2024-08-25T19:37:52.791 INFO:root:teuthology version: 1.2.1.dev2+g4d4c5e3 2024-08-25T19:37:52.806 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-25T19:37:52.896 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544 branch: main description: krbd/mirror/{bluestore-bitmap clusters/2-node conf install/ceph ms_mode$/{legacy} tasks/compare-mirror-images} email: ceph-qa@ceph.com first_in_suite: false job_id: '7870544' kernel: flavor: default kdb: 1 sha1: 8618c856c9a644ad80ca70c748bebe0c9b6227d4 last_in_suite: false machine_type: smithi meta: - desc: 2 ceph clusters with 1 mon, 1 mgr and 3 osd each name: teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: rbd default map options: ms_mode=legacy global: mon warn on pool no app: false ms die on skipped message: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: db4959e44f0252a53b4d8688fbf88b3dda70ea12 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: db4959e44f0252a53b4d8688fbf88b3dda70ea12 extra_system_packages: - pv thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: db4959e44f0252a53b4d8688fbf88b3dda70ea12 owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - cluster1.mon.a - cluster1.mgr.x - cluster1.osd.0 - cluster1.osd.1 - cluster1.osd.2 - - cluster2.mon.a - cluster2.mgr.x - cluster2.osd.0 - cluster2.osd.1 - cluster2.osd.2 - cluster1.client.mirror - cluster1.client.mirror.0 - cluster2.client.mirror - cluster2.client.mirror.0 seed: 7857 sha1: db4959e44f0252a53b4d8688fbf88b3dda70ea12 sleep_before_teardown: 0 subset: 0/4 suite: krbd suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_db4959e44f0252a53b4d8688fbf88b3dda70ea12/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: db4959e44f0252a53b4d8688fbf88b3dda70ea12 targets: smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLqR8VvVK77NDOLReMDLA0UunAoSi8mErvCIt51FxQdFsTnMvrtmPsn94raf9Z/ZxUUPgLrklOSIjbZn+LeZtSA= smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFSv2EVnz87750syph3/YS1Efgvmc3yIYvBNKk3Bi1s9Hk1w8k3ZHfg1lD5TTbZYeIWMXihSQDjR7bzAai1KHj8= tasks: - install: extra_packages: - rbd-mirror - ceph: cluster: cluster1 - ceph: cluster: cluster2 - rbd-mirror: client: cluster1.client.mirror.0 thrash: false - rbd-mirror: client: cluster2.client.mirror.0 thrash: false - workunit: clients: cluster1.client.mirror: - rbd/compare_mirror_images.sh env: RBD_DEVICE_TYPE: krbd RBD_MIRROR_USE_RBD_MIRROR: '1' timeout: 3h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 4d4c5e30374204f3b32f867e61bb4e1a3880f672 timestamp: 2024-08-24_20:48:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-25T19:37:52.896 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_db4959e44f0252a53b4d8688fbf88b3dda70ea12/qa; will attempt to use it 2024-08-25T19:37:52.897 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_db4959e44f0252a53b4d8688fbf88b3dda70ea12/qa/tasks 2024-08-25T19:37:52.897 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-25T19:37:52.899 INFO:teuthology.task.internal:Checking packages... 2024-08-25T19:37:52.920 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'db4959e44f0252a53b4d8688fbf88b3dda70ea12' 2024-08-25T19:37:52.920 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-25T19:37:52.920 INFO:teuthology.packaging:ref: None 2024-08-25T19:37:52.920 INFO:teuthology.packaging:tag: None 2024-08-25T19:37:52.920 INFO:teuthology.packaging:branch: main 2024-08-25T19:37:52.920 INFO:teuthology.packaging:sha1: db4959e44f0252a53b4d8688fbf88b3dda70ea12 2024-08-25T19:37:52.920 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-08-25T19:37:53.148 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-5971.gdb4959e4 2024-08-25T19:37:53.150 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-25T19:37:53.160 INFO:teuthology.task.internal:no buildpackages task found 2024-08-25T19:37:53.160 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-25T19:37:53.177 INFO:teuthology.task.internal:Saving configuration 2024-08-25T19:37:53.192 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-25T19:37:53.203 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-25T19:37:53.228 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-25 19:32:56.448493', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLqR8VvVK77NDOLReMDLA0UunAoSi8mErvCIt51FxQdFsTnMvrtmPsn94raf9Z/ZxUUPgLrklOSIjbZn+LeZtSA='} 2024-08-25T19:37:53.248 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-25 19:32:56.449894', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFSv2EVnz87750syph3/YS1Efgvmc3yIYvBNKk3Bi1s9Hk1w8k3ZHfg1lD5TTbZYeIWMXihSQDjR7bzAai1KHj8='} 2024-08-25T19:37:53.249 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-25T19:37:53.262 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2'] 2024-08-25T19:37:53.262 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.mgr.x', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster2.client.mirror', 'cluster2.client.mirror.0'] 2024-08-25T19:37:53.262 INFO:teuthology.run_tasks:Running task console_log... 2024-08-25T19:37:53.326 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f12b4fbfac0>, signals=[15]) 2024-08-25T19:37:53.326 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-25T19:37:53.332 INFO:teuthology.task.internal:Opening connections... 2024-08-25T19:37:53.332 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-08-25T19:37:53.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-25T19:37:53.411 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2024-08-25T19:37:53.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-25T19:37:53.486 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-25T19:37:53.491 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-08-25T19:37:53.510 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-08-25T19:37:53.510 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-08-25T19:37:53.568 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2024-08-25T19:37:53.568 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2024-08-25T19:37:53.569 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2024-08-25T19:37:53.569 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2024-08-25T19:37:53.569 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2024-08-25T19:37:53.569 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2024-08-25T19:37:53.569 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-25T19:37:53.569 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-08-25T19:37:53.569 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2024-08-25T19:37:53.569 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-25T19:37:53.569 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2024-08-25T19:37:53.569 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-25T19:37:53.569 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-25T19:37:53.570 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-25T19:37:53.570 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-08-25T19:37:53.595 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2024-08-25T19:37:53.613 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2024-08-25T19:37:53.613 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2024-08-25T19:37:53.672 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2024-08-25T19:37:53.672 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9" 2024-08-25T19:37:53.672 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2024-08-25T19:37:53.672 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2024-08-25T19:37:53.672 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9" 2024-08-25T19:37:53.672 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9" 2024-08-25T19:37:53.672 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-25T19:37:53.673 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2024-08-25T19:37:53.673 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon" 2024-08-25T19:37:53.673 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-25T19:37:53.673 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2024-08-25T19:37:53.673 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-25T19:37:53.673 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-25T19:37:53.673 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-25T19:37:53.673 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2024-08-25T19:37:53.695 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-25T19:37:53.707 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-25T19:37:53.714 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-25T19:37:53.714 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-08-25T19:37:53.717 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2024-08-25T19:37:53.734 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-25T19:37:53.742 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-25T19:37:53.742 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-08-25T19:37:53.774 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2024-08-25T19:37:53.858 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-25T19:37:53.917 INFO:teuthology.run_tasks:Running task kernel... 2024-08-25T19:37:53.932 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': '8618c856c9a644ad80ca70c748bebe0c9b6227d4'} 2024-08-25T19:37:53.932 INFO:teuthology.task.kernel:config {'cluster1.mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': '8618c856c9a644ad80ca70c748bebe0c9b6227d4'}, 'cluster2.mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': '8618c856c9a644ad80ca70c748bebe0c9b6227d4'}}, timeout 300 2024-08-25T19:37:53.933 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-25T19:37:53.933 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-25T19:37:53.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-25T19:37:53.950 DEBUG:teuthology.task.kernel:sha1 for cluster1.mon.a is 8618c856c9a644ad80ca70c748bebe0c9b6227d4 2024-08-25T19:37:53.950 INFO:teuthology.task.kernel:Checking kernel version of cluster1.mon.a, want "8618c856c9a644ad80ca70c748bebe0c9b6227d4"... 2024-08-25T19:37:53.951 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-08-25T19:37:53.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-25T19:37:53.952 DEBUG:teuthology.task.kernel:sha1 for cluster2.mon.a is 8618c856c9a644ad80ca70c748bebe0c9b6227d4 2024-08-25T19:37:53.952 INFO:teuthology.task.kernel:Checking kernel version of cluster2.mon.a, want "8618c856c9a644ad80ca70c748bebe0c9b6227d4"... 2024-08-25T19:37:53.952 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2024-08-25T19:37:54.011 INFO:teuthology.orchestra.run.smithi121.stdout:5.14.0-496.el9.x86_64 2024-08-25T19:37:54.011 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-496.el9.x86_64 vs 8618c856c9a644ad80ca70c748bebe0c9b6227d4 2024-08-25T19:37:54.012 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-08-25T19:37:54.012 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8618c856c9a644ad80ca70c748bebe0c9b6227d4 2024-08-25T19:37:54.015 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-496.el9.x86_64 2024-08-25T19:37:54.016 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-496.el9.x86_64 vs 8618c856c9a644ad80ca70c748bebe0c9b6227d4 2024-08-25T19:37:54.016 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-08-25T19:37:54.016 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8618c856c9a644ad80ca70c748bebe0c9b6227d4 2024-08-25T19:37:54.162 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum upgrade -y linux-firmware 2024-08-25T19:37:54.163 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum upgrade -y linux-firmware 2024-08-25T19:37:56.684 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-08-25T19:37:58.196 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.2 MB 00:02 2024-08-25T19:38:00.396 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-08-25T19:38:02.451 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 4.3 MB/s | 20 MB 00:04 2024-08-25T19:38:05.772 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.6 MB 00:01 2024-08-25T19:38:07.117 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.6 MB 00:00 2024-08-25T19:38:07.397 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 18 kB 00:00 2024-08-25T19:38:08.218 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 32 MB/s | 22 MB 00:00 2024-08-25T19:38:08.610 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 18 kB 00:00 2024-08-25T19:38:09.439 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 31 MB/s | 22 MB 00:00 2024-08-25T19:38:13.774 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-08-25T19:38:14.844 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-08-25T19:38:15.789 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-08-25T19:38:15.790 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2024-08-25T19:38:15.790 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-08-25T19:38:15.893 INFO:teuthology.task.kernel:Downloading kernel 8618c856c9a644ad80ca70c748bebe0c9b6227d4 on cluster1.mon.a... 2024-08-25T19:38:15.893 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8618c856c9a644ad80ca70c748bebe0c9b6227d4 2024-08-25T19:38:16.033 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/8618c856c9a644ad80ca70c748bebe0c9b6227d4/centos/9/flavors/default/x86_64/ 2024-08-25T19:38:16.033 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc3_g8618c856c9a6-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/8618c856c9a644ad80ca70c748bebe0c9b6227d4/centos/9/flavors/default/x86_64/ --input-file=- 2024-08-25T19:38:16.035 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi119.front.sepia.ceph.com to complete... 2024-08-25T19:38:16.883 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-08-25T19:38:16.884 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-08-25T19:38:16.884 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-08-25T19:38:17.001 INFO:teuthology.task.kernel:Downloading kernel 8618c856c9a644ad80ca70c748bebe0c9b6227d4 on cluster2.mon.a... 2024-08-25T19:38:17.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8618c856c9a644ad80ca70c748bebe0c9b6227d4 2024-08-25T19:38:17.138 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/8618c856c9a644ad80ca70c748bebe0c9b6227d4/centos/9/flavors/default/x86_64/ 2024-08-25T19:38:17.138 DEBUG:teuthology.orchestra.run.smithi121:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc3_g8618c856c9a6-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/8618c856c9a644ad80ca70c748bebe0c9b6227d4/centos/9/flavors/default/x86_64/ --input-file=- 2024-08-25T19:38:17.141 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi121.front.sepia.ceph.com to complete... 2024-08-25T19:38:30.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25 19:38:30 URL:https://2.chacra.ceph.com/r/kernel/testing/8618c856c9a644ad80ca70c748bebe0c9b6227d4/centos/9/flavors/default/x86_64/kernel-6.11.0_rc3_g8618c856c9a6-1.x86_64.rpm [360008351/360008351] -> "/tmp/kernel.x86_64.rpm" [1] 2024-08-25T19:38:30.035 INFO:teuthology.orchestra.run.smithi119.stderr:FINISHED --2024-08-25 19:38:30-- 2024-08-25T19:38:30.035 INFO:teuthology.orchestra.run.smithi119.stderr:Total wall clock time: 14s 2024-08-25T19:38:30.035 INFO:teuthology.orchestra.run.smithi119.stderr:Downloaded: 1 files, 343M in 13s (25.6 MB/s) 2024-08-25T19:38:30.038 INFO:teuthology.task.kernel:Installing kernel 8618c856c9a644ad80ca70c748bebe0c9b6227d4 on cluster1.mon.a... 2024-08-25T19:38:30.039 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-08-25T19:38:32.152 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying... ######################################## 2024-08-25T19:38:32.337 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing... ######################################## 2024-08-25T19:38:32.459 INFO:teuthology.orchestra.run.smithi119.stdout:Updating / installing... 2024-08-25T19:38:39.619 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25 19:38:39 URL:https://2.chacra.ceph.com/r/kernel/testing/8618c856c9a644ad80ca70c748bebe0c9b6227d4/centos/9/flavors/default/x86_64/kernel-6.11.0_rc3_g8618c856c9a6-1.x86_64.rpm [360008351/360008351] -> "/tmp/kernel.x86_64.rpm" [1] 2024-08-25T19:38:39.619 INFO:teuthology.orchestra.run.smithi121.stderr:FINISHED --2024-08-25 19:38:39-- 2024-08-25T19:38:39.619 INFO:teuthology.orchestra.run.smithi121.stderr:Total wall clock time: 22s 2024-08-25T19:38:39.619 INFO:teuthology.orchestra.run.smithi121.stderr:Downloaded: 1 files, 343M in 22s (15.6 MB/s) 2024-08-25T19:38:39.621 INFO:teuthology.task.kernel:Installing kernel 8618c856c9a644ad80ca70c748bebe0c9b6227d4 on cluster2.mon.a... 2024-08-25T19:38:39.622 DEBUG:teuthology.orchestra.run.smithi121:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-08-25T19:38:40.404 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-6.11.0_rc3_g8618c856c9a6-1 ######################################## 2024-08-25T19:38:41.773 INFO:teuthology.orchestra.run.smithi121.stdout:Verifying... ######################################## 2024-08-25T19:38:42.028 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing... ######################################## 2024-08-25T19:38:42.142 INFO:teuthology.orchestra.run.smithi121.stdout:Updating / installing... 2024-08-25T19:38:50.114 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-6.11.0_rc3_g8618c856c9a6-1 ######################################## 2024-08-25T19:39:20.121 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-08-25T19:39:20.122 DEBUG:teuthology.orchestra.run.smithi119:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-08-25T19:39:20.196 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/System.map-6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:20.196 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/config-6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:20.196 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/vmlinuz-6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:20.196 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:20.196 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/System.map 2024-08-25T19:39:20.196 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/config 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/crc32c-intel.ko 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-08-25T19:39:20.197 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/serpent-avx2.ko 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kernel 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kernel/cpu 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kernel/cpu/mce 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-08-25T19:39:20.198 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kvm 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kvm/kvm-amd.ko 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kvm/kvm-intel.ko 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kvm/kvm.ko 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/ansi_cprng.ko 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/anubis.ko 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/arc4.ko 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx/async_memcpy.ko 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx/async_pq.ko 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx/async_raid6_recov.ko 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx/async_tx.ko 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx/async_xor.ko 2024-08-25T19:39:20.199 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx/raid6test.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/authenc.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/authencesn.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/blake2b_generic.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/blowfish_common.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/blowfish_generic.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/camellia_generic.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/cast5_generic.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/cast6_generic.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/cast_common.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/ccm.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/cmac.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/crc32_generic.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/crypto_engine.ko 2024-08-25T19:39:20.200 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/crypto_user.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/cts.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/des_generic.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/dh_generic.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/ecc.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/ecdh_generic.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/echainiv.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/essiv.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/fcrypt.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/khazad.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/lz4.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/lz4hc.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/md4.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/michael_mic.ko 2024-08-25T19:39:20.201 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/pcbc.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/pcrypt.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/rmd160.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/seed.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/serpent_generic.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/tcrypt.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/tea.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/twofish_common.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/twofish_generic.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/vmac.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/wp512.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/xcbc.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/xor.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/xxhash_generic.ko 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers 2024-08-25T19:39:20.202 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/acpi_ipmi.ko 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/acpi_pad.ko 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/ec_sys.ko 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/platform_profile.ko 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/sbs.ko 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/sbshc.ko 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/video.ko 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/acard-ahci.ko 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/ahci_platform.ko 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/ata_generic.ko 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/libahci_platform.ko 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_acpi.ko 2024-08-25T19:39:20.203 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_ali.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_amd.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_artop.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_atiixp.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_atp867x.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_cmd640.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_cmd64x.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_cypress.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_efar.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_hpt366.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_hpt37x.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_hpt3x2n.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_hpt3x3.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_it8213.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_it821x.ko 2024-08-25T19:39:20.204 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_jmicron.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_marvell.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_mpiix.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_netcell.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_ninja32.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_ns87410.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_ns87415.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_oldpiix.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_opti.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_optidma.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_pcmcia.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_pdc2027x.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_piccolo.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_rdc.ko 2024-08-25T19:39:20.205 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_sch.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_serverworks.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_sil680.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_sis.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_sl82c105.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_triflex.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_via.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pdc_adma.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_inic162x.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_mv.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_nv.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_promise.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_qstor.ko 2024-08-25T19:39:20.206 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_sil.ko 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_sil24.ko 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_sis.ko 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_svw.ko 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_sx4.ko 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_uli.ko 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_via.ko 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_vsc.ko 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/auxdisplay 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/auxdisplay/ks0108.ko 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/base 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/base/regmap 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/base/regmap/regmap-i2c.ko 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/base/regmap/regmap-mmio.ko 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/bcma 2024-08-25T19:39:20.207 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/bcma/bcma.ko 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/aoe 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/aoe/aoe.ko 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/brd.ko 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/drbd 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/drbd/drbd.ko 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/floppy.ko 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/loop.ko 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/mtip32xx 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/nbd.ko 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/null_blk 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/null_blk/null_blk.ko 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/pktcdvd.ko 2024-08-25T19:39:20.208 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/rbd.ko 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/virtio_blk.ko 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/xen-blkback 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/xen-blkfront.ko 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/zram 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/zram/zram.ko 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hangcheck-timer.ko 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hw_random 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hw_random/amd-rng.ko 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hw_random/intel-rng.ko 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hw_random/via-rng.ko 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hw_random/virtio-rng.ko 2024-08-25T19:39:20.209 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi/ipmi_si.ko 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/lp.ko 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/mwave 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/mwave/mwave.ko 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ppdev.ko 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tlclk.ko 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm 2024-08-25T19:39:20.210 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm/tpm.ko 2024-08-25T19:39:20.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm/tpm_atmel.ko 2024-08-25T19:39:20.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm/tpm_infineon.ko 2024-08-25T19:39:20.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm/tpm_nsc.ko 2024-08-25T19:39:20.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm/tpm_tis.ko 2024-08-25T19:39:20.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-08-25T19:39:20.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/uv_mmtimer.ko 2024-08-25T19:39:20.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/virtio_console.ko 2024-08-25T19:39:20.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq 2024-08-25T19:39:20.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-08-25T19:39:20.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-08-25T19:39:20.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq/p4-clockmod.ko 2024-08-25T19:39:20.211 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq/powernow-k8.ko 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq/speedstep-lib.ko 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/ccp 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/ccp/ccp.ko 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/intel 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/intel/qat 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/intel/qat/qat_common 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/padlock-aes.ko 2024-08-25T19:39:20.212 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/padlock-sha.ko 2024-08-25T19:39:20.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/virtio 2024-08-25T19:39:20.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-08-25T19:39:20.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dca 2024-08-25T19:39:20.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dca/dca.ko 2024-08-25T19:39:20.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/devfreq 2024-08-25T19:39:20.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-08-25T19:39:20.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dma 2024-08-25T19:39:20.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dma/dw 2024-08-25T19:39:20.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dma/dw/dw_dmac.ko 2024-08-25T19:39:20.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dma/ioat 2024-08-25T19:39:20.213 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dma/ioat/ioatdma.ko 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/amd64_edac.ko 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/e752x_edac.ko 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/edac_mce_amd.ko 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i3000_edac.ko 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i3200_edac.ko 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i5100_edac.ko 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i5400_edac.ko 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i7300_edac.ko 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i7core_edac.ko 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i82975x_edac.ko 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/ie31200_edac.ko 2024-08-25T19:39:20.214 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/sb_edac.ko 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/x38_edac.ko 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/firmware 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/firmware/edd.ko 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/firmware/iscsi_ibft.ko 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/gpio 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/gpio/gpio-ich.ko 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/gpio/gpio-viperboard.ko 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-appleir.ko 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-aureal.ko 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-axff.ko 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-dr.ko 2024-08-25T19:39:20.215 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-elecom.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-elo.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-emsff.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-gaff.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-gt683r.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-gyration.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-holtek-kbd.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-holtek-mouse.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-holtekff.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-hyperv.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-icade.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-keytouch.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-kye.ko 2024-08-25T19:39:20.216 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-lcpower.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-led.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-lenovo.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-logitech-dj.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-multitouch.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-ortek.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-penmount.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-petalynx.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-picolcd.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-pl.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-plantronics.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-primax.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-rmi.ko 2024-08-25T19:39:20.217 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-arvo.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-common.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-isku.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-kone.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-konepure.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-lua.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-pyra.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-ryos.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-savu.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-saitek.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-samsung.ko 2024-08-25T19:39:20.218 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-sensor-hub.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-sjoy.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-sony.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-speedlink.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-steelseries.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-sunplus.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-thrustmaster.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-tivo.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-tmff.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-topseed.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-twinhan.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-uclogic.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-waltop.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-wiimote.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-xinmo.ko 2024-08-25T19:39:20.219 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-zpff.ko 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-zydacron.ko 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/uhid.ko 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/wacom.ko 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hv 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hv/hv_balloon.ko 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hv/hv_utils.ko 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hv/hv_vmbus.ko 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/abituguru.ko 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/abituguru3.ko 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/acpi_power_meter.ko 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ad7414.ko 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ad7418.ko 2024-08-25T19:39:20.220 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adc128d818.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adm1025.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adm1026.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adm1029.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adm1031.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adm9240.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ads7828.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adt7410.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adt7411.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adt7462.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adt7470.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adt7475.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adt7x10.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/amc6821.ko 2024-08-25T19:39:20.221 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/applesmc.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/asb100.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/asc7621.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/asus_atk0110.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/atxp1.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/coretemp.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/dme1737.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ds1621.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ds620.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/emc1403.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/emc6w201.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/f71805f.ko 2024-08-25T19:39:20.222 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/f71882fg.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/f75375s.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/fam15h_power.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/fschmd.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/g760a.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/g762.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/gl518sm.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/gl520sm.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/hwmon-vid.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/i5500_temp.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/i5k_amb.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ibmaem.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ibmpex.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ina209.ko 2024-08-25T19:39:20.223 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ina2xx.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/it87.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/k10temp.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/k8temp.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lineage-pem.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm63.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm73.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm75.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm77.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm78.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm80.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm83.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm85.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm87.ko 2024-08-25T19:39:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm90.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm92.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm93.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm95234.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm95241.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm95245.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc2945.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc4151.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc4215.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc4222.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc4245.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc4260.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc4261.ko 2024-08-25T19:39:20.225 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max16065.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max1619.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max1668.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max197.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max6639.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max6650.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max6697.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/mcp3021.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/nct6683.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/nct6775-core.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/nct6775.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/nct7802.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pc87360.ko 2024-08-25T19:39:20.226 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pc87427.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pcf8591.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/max16064.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/max34440.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/max8688.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-08-25T19:39:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/powr1220.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/sch5627.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/sch5636.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/sch56xx-common.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/sht15.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/sht21.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/shtc1.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/sis5595.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/smsc47b397.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/smsc47m1.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/smsc47m192.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/thmc50.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/tmp102.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/tmp103.ko 2024-08-25T19:39:20.228 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/tmp401.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/tmp421.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/via-cputemp.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/via686a.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/vt1211.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/vt8231.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83627ehf.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83627hf.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83781d.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83791d.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83792d.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83793.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83795.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83l785ts.ko 2024-08-25T19:39:20.229 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83l786ng.ko 2024-08-25T19:39:20.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c 2024-08-25T19:39:20.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/algos 2024-08-25T19:39:20.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-08-25T19:39:20.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-08-25T19:39:20.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses 2024-08-25T19:39:20.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-08-25T19:39:20.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-08-25T19:39:20.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-08-25T19:39:20.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-08-25T19:39:20.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-08-25T19:39:20.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-08-25T19:39:20.230 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-i801.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-isch.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-parport.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-08-25T19:39:20.231 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-via.ko 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/i2c-dev.ko 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/i2c-mux.ko 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/i2c-smbus.ko 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/i2c-stub.ko 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/ib_cm.ko 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/ib_core.ko 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/ib_umad.ko 2024-08-25T19:39:20.232 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/iw_cm.ko 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/rdma_cm.ko 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/cxgb4 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/mlx4 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/mlx5 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/mthca 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-08-25T19:39:20.233 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/ipoib 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/iser 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/isert 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/srp 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/srpt 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/gameport 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-08-25T19:39:20.234 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/gameport/fm801-gp.ko 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/gameport/gameport.ko 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/joydev.ko 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/keyboard 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/keyboard/gpio_keys.ko 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/appletouch.ko 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/bcm5974.ko 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/cyapatp.ko 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/elan_i2c.ko 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/sermouse.ko 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/synaptics_usb.ko 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/vsxxxaa.ko 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/rmi4 2024-08-25T19:39:20.235 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/rmi4/rmi_core.ko 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/serio 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/serio/altera_ps2.ko 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/serio/arc_ps2.ko 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/serio/serio_raw.ko 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/sparse-keymap.ko 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/leds-blinkm.ko 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/leds-lm3530.ko 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/leds-lp3944.ko 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/leds-ss4200.ko 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger 2024-08-25T19:39:20.236 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/bcache 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/bcache/bcache.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-bio-prison.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-cache-smq.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-cache.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-crypt.ko 2024-08-25T19:39:20.237 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-delay.ko 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-flakey.ko 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-log-userspace.ko 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-multipath.ko 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-queue-length.ko 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-raid.ko 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-round-robin.ko 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-service-time.ko 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-switch.ko 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-thin-pool.ko 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-verity.ko 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/persistent-data 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-08-25T19:39:20.238 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/raid0.ko 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/raid1.ko 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/raid10.ko 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/raid456.ko 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptbase.ko 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptctl.ko 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptfc.ko 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptlan.ko 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptsas.ko 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptscsih.ko 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptspi.ko 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd 2024-08-25T19:39:20.239 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/lpc_ich.ko 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/lpc_sch.ko 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/mfd-core.ko 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/sm501.ko 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/viperboard.ko 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/vx855.ko 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/wl1273-core.ko 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/altera-stapl 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/apds9802als.ko 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/apds990x.ko 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/bh1770glc.ko 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/cb710 2024-08-25T19:39:20.240 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/cb710/cb710.ko 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/eeprom 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/eeprom/at24.ko 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/eeprom/max6875.ko 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/enclosure.ko 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/hpilo.ko 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/isl29003.ko 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/isl29020.ko 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/lis3lv02d 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/mei 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/mei/mei-me.ko 2024-08-25T19:39:20.241 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/mei/mei-txe.ko 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/mei/mei.ko 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/sgi-gru 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/sgi-gru/gru.ko 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/sgi-xp 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/sgi-xp/xp.ko 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/sgi-xp/xpc.ko 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/tifm_7xx1.ko 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/tifm_core.ko 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/tsl2550.ko 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/vmw_balloon.ko 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/vmw_vmci 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-08-25T19:39:20.242 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/bonding 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/bonding/bonding.ko 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa/b53 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa/b53/b53_common.ko 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa/bcm-sf2.ko 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa/mv88e6060.ko 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa/mv88e6xxx 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dummy.ko 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/eql.ko 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/3com 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-08-25T19:39:20.243 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/8390 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/8390/8390.ko 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/adaptec 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/agere 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/agere/et131x.ko 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/alteon 2024-08-25T19:39:20.244 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/altera 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/amd 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/alx 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atl1c 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atl1e 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-08-25T19:39:20.245 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atlx 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/genet 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/brocade 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/brocade/bna 2024-08-25T19:39:20.246 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/cisco 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/cisco/enic 2024-08-25T19:39:20.247 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dlink 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dnet.ko 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/emulex 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/emulex/benet 2024-08-25T19:39:20.248 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/ethoc.ko 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/fealnx.ko 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/e100.ko 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/e1000 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/e1000e 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/fm10k 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/i40e 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/iavf 2024-08-25T19:39:20.249 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/igb 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/igbvf 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/ixgbe 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/ixgbevf 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/libeth 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/libie 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/jme.ko 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/marvell 2024-08-25T19:39:20.250 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/marvell/skge.ko 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox/mlx4 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox/mlx5 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/micrel 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/myricom 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/myricom/myri10ge 2024-08-25T19:39:20.251 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/natsemi 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/neterion 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/nvidia 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/packetengines 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/qlogic 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/qlogic/netxen 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-08-25T19:39:20.252 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/rdc 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/realtek 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/realtek/atp.ko 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/rocker 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sfc 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/silan 2024-08-25T19:39:20.253 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sis 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sis/sis190.ko 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sis/sis900.ko 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/smsc 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/stmicro 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/stmicro/stmmac 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sun 2024-08-25T19:39:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sun/cassini.ko 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sun/niu.ko 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sun/sungem.ko 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/tehuti 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/ti 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/ti/tlan.ko 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/via 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/wiznet 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-08-25T19:39:20.255 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/xircom 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/geneve.ko 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/hyperv 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ieee802154 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ieee802154/fakelb.ko 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ifb.ko 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ipvlan 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ipvlan/ipvlan.ko 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/macvlan.ko 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/macvtap.ko 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/mdio 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/mdio.ko 2024-08-25T19:39:20.256 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/mdio/mdio-regmap.ko 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/mii.ko 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/net_failover.ko 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/netconsole.ko 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/nlmon.ko 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/pcs 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/pcs/pcs-lynx.ko 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/amd.ko 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-08-25T19:39:20.257 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/bcm7xxx.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/bcm87xx.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/broadcom.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/cicada.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/davicom.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/dp83640.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/et1011c.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/icplus.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/lxt.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/marvell.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/micrel.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/national.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/phylink.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/qsemi.ko 2024-08-25T19:39:20.258 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/realtek.ko 2024-08-25T19:39:20.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/smsc.ko 2024-08-25T19:39:20.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/ste10Xp.ko 2024-08-25T19:39:20.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/vitesse.ko 2024-08-25T19:39:20.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp 2024-08-25T19:39:20.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/bsd_comp.ko 2024-08-25T19:39:20.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/ppp_async.ko 2024-08-25T19:39:20.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/ppp_deflate.ko 2024-08-25T19:39:20.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/ppp_generic.ko 2024-08-25T19:39:20.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/ppp_mppe.ko 2024-08-25T19:39:20.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/ppp_synctty.ko 2024-08-25T19:39:20.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/pppoe.ko 2024-08-25T19:39:20.259 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/pppox.ko 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/pptp.ko 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/slip 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/slip/slhc.ko 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/slip/slip.ko 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/sungem_phy.ko 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/tap.ko 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team/team.ko 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team/team_mode_activebackup.ko 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team/team_mode_broadcast.ko 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team/team_mode_random.ko 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-08-25T19:39:20.260 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/tun.ko 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/veth.ko 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/virtio_net.ko 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/vmxnet3 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/vxlan 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/vxlan/vxlan.ko 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/xen-netback 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/xen-netback/xen-netback.ko 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/xen-netfront.ko 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ntb 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ntb/ntb.ko 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/nvme 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/nvme/host 2024-08-25T19:39:20.261 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/nvme/host/nvme-core.ko 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/nvme/host/nvme.ko 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/parport 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/parport/parport.ko 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/parport/parport_cs.ko 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/parport/parport_pc.ko 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/parport/parport_serial.ko 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pci 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pci/hotplug 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pci/pcie 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pci/pcie/aer_inject.ko 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pci/xen-pcifront.ko 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pcmcia 2024-08-25T19:39:20.262 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pcmcia/i82092.ko 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pcmcia/pd6729.ko 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pcmcia/yenta_socket.ko 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pinctrl 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pinctrl/intel 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/acer-wmi.ko 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/acerhdf.ko 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/apple-gmux.ko 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/asus-laptop.ko 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/asus-wmi.ko 2024-08-25T19:39:20.263 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/classmate-laptop.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/hdaps.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/intel 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/intel_ips.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/msi-wmi.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/mxm-wmi.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/samsung-laptop.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/samsung-q10.ko 2024-08-25T19:39:20.264 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/topstar-laptop.ko 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/toshiba_haps.ko 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/wmi-bmof.ko 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/wmi.ko 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/powercap 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/powercap/intel_rapl_common.ko 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/powercap/intel_rapl_msr.ko 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pps 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pps/clients 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pps/clients/pps-gpio.ko 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pps/clients/pps-ldisc.ko 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pps/clients/pps_parport.ko 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pps/pps_core.ko 2024-08-25T19:39:20.265 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ptp 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ptp/ptp.ko 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ptp/ptp_kvm.ko 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ras 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ras/amd 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ras/amd/atl 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ras/amd/fmpm.ko 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-bq32k.ko 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1286.ko 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1307.ko 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1374.ko 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1511.ko 2024-08-25T19:39:20.266 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1553.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1672.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1742.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds2404.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds3232.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-em3027.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-fm3130.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-isl12022.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-isl1208.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-m41t80.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-m48t35.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-m48t59.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-max6900.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-msm6242.ko 2024-08-25T19:39:20.267 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-pcf2127.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-pcf85063.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-pcf8523.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-pcf8563.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-pcf8583.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-rp5c01.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-rs5c372.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-rx8025.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-rx8581.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-x1205.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/3w-9xxx.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/3w-sas.ko 2024-08-25T19:39:20.268 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/3w-xxxx.ko 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/BusLogic.ko 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/a100u2w.ko 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/aacraid 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/aacraid/aacraid.ko 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/advansys.ko 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/aic7xxx 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/am53c974.ko 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/arcmsr 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/atp870u.ko 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/be2iscsi 2024-08-25T19:39:20.269 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/bfa 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/bfa/bfa.ko 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/bnx2fc 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/bnx2i 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/ch.ko 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/csiostor 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/csiostor/csiostor.ko 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/cxgbi 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/cxgbi/cxgb3i 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/cxgbi/cxgb4i 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-08-25T19:39:20.270 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/dc395x.ko 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/device_handler 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/dmx3191d.ko 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/esas2r 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/esas2r/esas2r.ko 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/esp_scsi.ko 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/fcoe 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/fcoe/fcoe.ko 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-08-25T19:39:20.271 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/fnic 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/fnic/fnic.ko 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/hpsa.ko 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/hptiop.ko 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/hv_storvsc.ko 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/initio.ko 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/ipr.ko 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/ips.ko 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/isci 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/isci/isci.ko 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/iscsi_tcp.ko 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/libfc 2024-08-25T19:39:20.272 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/libfc/libfc.ko 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/libiscsi.ko 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/libiscsi_tcp.ko 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/libsas 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/libsas/libsas.ko 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/lpfc 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/lpfc/lpfc.ko 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/megaraid 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/megaraid.ko 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/mpt3sas 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-08-25T19:39:20.273 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/mvsas 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/mvsas/mvsas.ko 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/mvumi.ko 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/pm8001 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/pmcraid.ko 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/qla1280.ko 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/qla2xxx 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/qla4xxx 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/raid_class.ko 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/scsi_debug.ko 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/scsi_transport_fc.ko 2024-08-25T19:39:20.274 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/scsi_transport_sas.ko 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/scsi_transport_spi.ko 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/scsi_transport_srp.ko 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/ses.ko 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/st.ko 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/stex.ko 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/sym53c8xx_2 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/virtio_scsi.ko 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/vmw_pvscsi.ko 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/wd719x.ko 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ssb 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ssb/ssb.ko 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target 2024-08-25T19:39:20.275 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/iscsi 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/loopback 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/loopback/tcm_loop.ko 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/target_core_file.ko 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/target_core_iblock.ko 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/target_core_mod.ko 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/target_core_pscsi.ko 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/target_core_user.ko 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/tcm_fc 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel 2024-08-25T19:39:20.276 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-08-25T19:39:20.277 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/ipwireless 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/n_gsm.ko 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/n_hdlc.ko 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/nozomi.ko 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/serial 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/serial/8250 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/serial/arc_uart.ko 2024-08-25T19:39:20.278 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/serial/jsm 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/serial/jsm/jsm.ko 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ufs 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ufs/core 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ufs/core/ufshcd-core.ko 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ufs/host 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/uio 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/uio/uio.ko 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/uio/uio_aec.ko 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/uio/uio_cif.ko 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/uio/uio_pci_generic.ko 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/uio/uio_sercos3.ko 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb 2024-08-25T19:39:20.279 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/class 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/class/cdc-acm.ko 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/class/cdc-wdm.ko 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/class/usblp.ko 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/class/usbtmc.ko 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/host 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/host/sl811-hcd.ko 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/image 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/image/mdc800.ko 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/image/microtek.ko 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/adutux.ko 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/appledisplay.ko 2024-08-25T19:39:20.280 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/emi26.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/emi62.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/ezusb.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/idmouse.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/iowarrior.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/isight_firmware.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/ldusb.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/legousbtower.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/sisusbvga 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/trancevibrator.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/usb3503.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/usblcd.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/usbsevseg.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/uss720.ko 2024-08-25T19:39:20.281 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/yurex.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/phy 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/phy/phy-generic.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/aircable.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ark3116.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/belkin_sa.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ch341.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/cp210x.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/cyberjack.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/cypress_m8.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/digi_acceleport.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/empeg.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ftdi_sio.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/garmin_gps.ko 2024-08-25T19:39:20.282 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/io_edgeport.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/io_ti.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ipaq.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ipw.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ir-usb.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/keyspan.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/keyspan_pda.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/kl5kusb105.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/kobil_sct.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/mct_u232.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/mos7720.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/mos7840.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/navman.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/omninet.ko 2024-08-25T19:39:20.283 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/opticon.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/option.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/oti6858.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/pl2303.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/qcaux.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/qcserial.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/quatech2.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/safe_serial.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/sierra.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/spcp8x5.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ssu100.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/symbolserial.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/usb_debug.ko 2024-08-25T19:39:20.284 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/usb_wwan.ko 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/visor.ko 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/whiteheat.ko 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/xsens_mt.ko 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/uas.ko 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-alauda.ko 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-cypress.ko 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-datafab.ko 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-freecom.ko 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-isd200.ko 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-08-25T19:39:20.285 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-karma.ko 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-onetouch.ko 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-realtek.ko 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-sddr09.ko 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-sddr55.ko 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-usbat.ko 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/usb-storage.ko 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/usbip 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/usbip/usbip-core.ko 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/usbip/usbip-host.ko 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vfio 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vfio/pci 2024-08-25T19:39:20.286 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vfio/pci/vfio-pci.ko 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vfio/vfio.ko 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vhost 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vhost/vhost.ko 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vhost/vhost_iotlb.ko 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vhost/vhost_net.ko 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vhost/vhost_scsi.ko 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/video 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/video/backlight 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/video/backlight/apple_bl.ko 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/video/backlight/lcd.ko 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/video/backlight/platform_lcd.ko 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio 2024-08-25T19:39:20.287 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio_balloon.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio_mmio.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio_pci.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio_ring.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/masters 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/masters/ds2482.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/masters/ds2490.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-08-25T19:39:20.288 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_smem.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_therm.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/wire.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/alim1535_wdt.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/alim7101_wdt.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/f71808e_wdt.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/hpwdt.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/i6300esb.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-08-25T19:39:20.289 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/iTCO_wdt.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/ib700wdt.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/ibmasr.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/it8712f_wdt.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/it87_wdt.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/machzwd.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/nv_tco.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/pcwd_pci.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/pcwd_usb.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/sch311x_wdt.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/softdog.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/sp5100_tco.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/via_wdt.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-08-25T19:39:20.290 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/w83877f_wdt.ko 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/w83977f_wdt.ko 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/wdt_pci.ko 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/xen_wdt.ko 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-acpi-processor.ko 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-evtchn.ko 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-gntalloc.ko 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-gntdev.ko 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-pciback 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-privcmd.ko 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xenfs 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xenfs/xenfs.ko 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/9p 2024-08-25T19:39:20.291 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/9p/9p.ko 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/affs 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/affs/affs.ko 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/befs 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/befs/befs.ko 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/binfmt_misc.ko 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/btrfs 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/btrfs/btrfs.ko 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/cachefiles 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/cachefiles/cachefiles.ko 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ceph 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ceph/ceph.ko 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/coda 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/coda/coda.ko 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/cramfs 2024-08-25T19:39:20.292 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/cramfs/cramfs.ko 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/dlm 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/dlm/dlm.ko 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ecryptfs 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ecryptfs/ecryptfs.ko 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/f2fs 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/f2fs/f2fs.ko 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fat 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fat/fat.ko 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fat/msdos.ko 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fat/vfat.ko 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fuse 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fuse/cuse.ko 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fuse/fuse.ko 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/gfs2 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/gfs2/gfs2.ko 2024-08-25T19:39:20.293 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/hfs 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/hfs/hfs.ko 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/hfsplus 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/hfsplus/hfsplus.ko 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/isofs 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/isofs/isofs.ko 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/jfs 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/jfs/jfs.ko 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/lockd 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/lockd/lockd.ko 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/minix 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/minix/minix.ko 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/netfs 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/netfs/netfs.ko 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs 2024-08-25T19:39:20.294 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/blocklayout 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/filelayout 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/flexfilelayout 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/nfs.ko 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/nfsv3.ko 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/nfsv4.ko 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs_common 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs_common/grace.ko 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs_common/nfs_acl.ko 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfsd 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfsd/nfsd.ko 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nilfs2 2024-08-25T19:39:20.295 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nilfs2/nilfs2.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-celtic.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-centeuro.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-croatian.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-cyrillic.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-gaelic.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-greek.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-iceland.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-inuit.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-roman.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-romanian.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-turkish.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp1250.ko 2024-08-25T19:39:20.296 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp1251.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp1255.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp737.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp775.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp850.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp852.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp855.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp857.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp860.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp861.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp862.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp863.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp864.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp865.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp866.ko 2024-08-25T19:39:20.297 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp869.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp874.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp932.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp936.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp949.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp950.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_euc-jp.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-1.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-13.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-14.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-15.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-2.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-3.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-4.ko 2024-08-25T19:39:20.298 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-5.ko 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-6.ko 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-7.ko 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-9.ko 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_koi8-r.ko 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_koi8-ru.ko 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_koi8-u.ko 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_ucs2_utils.ko 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_utf8.ko 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/cluster 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/dlm 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/dlmfs 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-08-25T19:39:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/ocfs2.ko 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/overlayfs 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/overlayfs/overlay.ko 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/pstore 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/pstore/ramoops.ko 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/reiserfs 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/reiserfs/reiserfs.ko 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/romfs 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/romfs/romfs.ko 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/smb 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/smb/client 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/smb/client/cifs.ko 2024-08-25T19:39:20.300 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/smb/common 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/smb/common/cifs_arc4.ko 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/smb/common/cifs_md4.ko 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/squashfs 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/squashfs/squashfs.ko 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/sysv 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/sysv/sysv.ko 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/udf 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/udf/udf.ko 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ufs 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ufs/ufs.ko 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/xfs 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/xfs/xfs.ko 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/kernel 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/kernel/rcu 2024-08-25T19:39:20.301 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/kernel/rcu/rcutorture.ko 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/kernel/torture.ko 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/kernel/trace 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/asn1_encoder.ko 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/crc-itu-t.ko 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/crc8.ko 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/crypto 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/crypto/libaescfb.ko 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/crypto/libarc4.ko 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/crypto/libdes.ko 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/libcrc32c.ko 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/lru_cache.ko 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/lz4 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/lz4/lz4_compress.ko 2024-08-25T19:39:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/lz4/lz4hc_compress.ko 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/math 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/math/cordic.ko 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/raid6 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/raid6/raid6_pq.ko 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/reed_solomon 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/reed_solomon/reed_solomon.ko 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/ts_bm.ko 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/ts_fsm.ko 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/ts_kmp.ko 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/zstd 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/zstd/zstd_compress.ko 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/mm 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/mm/hwpoison-inject.ko 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net 2024-08-25T19:39:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/6lowpan.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_dest.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_fragment.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_hop.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_ipv6.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_mobility.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_routing.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_udp.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/802 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/802/garp.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/802/mrp.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/802/p8022.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/802/psnap.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/802/stp.ko 2024-08-25T19:39:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/8021q 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/8021q/8021q.ko 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/9p 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/9p/9pnet.ko 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/9p/9pnet_fd.ko 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/9p/9pnet_rdma.ko 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/9p/9pnet_virtio.ko 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/batman-adv 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/batman-adv/batman-adv.ko 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/br_netfilter.ko 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/bridge.ko 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_among.ko 2024-08-25T19:39:20.305 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_arp.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_ip.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_limit.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_log.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_mark.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_snat.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_stp.ko 2024-08-25T19:39:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebtables.ko 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ceph 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ceph/libceph.ko 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/core 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/core/failover.ko 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/core/pktgen.ko 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dccp 2024-08-25T19:39:20.307 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dccp/dccp.ko 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dccp/dccp_diag.ko 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dccp/dccp_ipv4.ko 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dccp/dccp_ipv6.ko 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dns_resolver 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dns_resolver/dns_resolver.ko 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dsa 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dsa/dsa_core.ko 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dsa/tag_brcm.ko 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dsa/tag_dsa.ko 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dsa/tag_none.ko 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dsa/tag_trailer.ko 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ieee802154 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ieee802154/6lowpan 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-08-25T19:39:20.308 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ieee802154/ieee802154.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ieee802154/ieee802154_socket.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/ah4.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/esp4.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/fou.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/gre.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/inet_diag.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/ip_gre.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/ip_tunnel.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/ip_vti.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/ipcomp.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/ipip.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/arp_tables.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-08-25T19:39:20.309 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/iptable_security.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-08-25T19:39:20.310 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_bic.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_dctcp.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_diag.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_highspeed.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_htcp.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_hybla.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_illinois.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_lp.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_scalable.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_vegas.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_veno.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_westwood.ko 2024-08-25T19:39:20.311 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_yeah.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tunnel4.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/udp_diag.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/udp_tunnel.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/xfrm4_tunnel.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/ah6.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/esp6.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/fou6.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/ip6_tunnel.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/ip6_vti.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/ipcomp6.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-08-25T19:39:20.312 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-08-25T19:39:20.313 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/sit.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/tunnel6.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/xfrm6_tunnel.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/key 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/key/af_key.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_core.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_debugfs.ko 2024-08-25T19:39:20.314 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_eth.ko 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_ip.ko 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_ip6.ko 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_netlink.ko 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_ppp.ko 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/llc 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/llc/llc.ko 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/mac802154 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/mac802154/mac802154.ko 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/mpls 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/mpls/mpls_gso.ko 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set.ko 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-08-25T19:39:20.315 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-08-25T19:39:20.316 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-08-25T19:39:20.317 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conncount.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_h323.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_irc.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_sane.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_sip.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-08-25T19:39:20.318 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_dup_netdev.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_flow_table.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_flow_table_inet.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_log_syslog.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_nat.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_nat_amanda.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_nat_ftp.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_nat_irc.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_nat_sip.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_nat_tftp.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_synproxy_core.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_tables.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nfnetlink.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nfnetlink_acct.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nfnetlink_log.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nfnetlink_osf.ko 2024-08-25T19:39:20.319 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nfnetlink_queue.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_chain_nat.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_compat.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_ct.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_dup_netdev.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_fib.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_fwd_netdev.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_hash.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_limit.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_log.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_masq.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_nat.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_numgen.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_queue.ko 2024-08-25T19:39:20.320 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_quota.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_redir.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_reject.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_reject_inet.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_socket.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_tunnel.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_AUDIT.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_CHECKSUM.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_CLASSIFY.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_CT.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_DSCP.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_HL.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_HMARK.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_IDLETIMER.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_LED.ko 2024-08-25T19:39:20.321 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_LOG.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_MASQUERADE.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_NETMAP.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_NFLOG.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_NFQUEUE.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_RATEEST.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_REDIRECT.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_SECMARK.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_TCPMSS.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_TEE.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_TPROXY.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_TRACE.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_addrtype.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_bpf.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_cgroup.ko 2024-08-25T19:39:20.322 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_cluster.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_comment.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_connbytes.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_connlabel.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_connlimit.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_connmark.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_conntrack.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_cpu.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_dccp.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_devgroup.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_dscp.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_ecn.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_esp.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_hashlimit.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_helper.ko 2024-08-25T19:39:20.323 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_hl.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_ipcomp.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_iprange.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_ipvs.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_l2tp.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_length.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_limit.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_mac.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_mark.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_multiport.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_nat.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_nfacct.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_osf.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_owner.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_physdev.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_pkttype.ko 2024-08-25T19:39:20.324 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_policy.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_quota.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_rateest.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_realm.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_recent.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_sctp.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_set.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_socket.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_state.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_statistic.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_string.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_tcpmss.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_time.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_u32.ko 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netlink 2024-08-25T19:39:20.325 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netlink/netlink_diag.ko 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/nsh 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/nsh/nsh.ko 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/openvswitch 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/openvswitch/openvswitch.ko 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/openvswitch/vport-geneve.ko 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/openvswitch/vport-gre.ko 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/openvswitch/vport-vxlan.ko 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/packet 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/packet/af_packet_diag.ko 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/rds 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/rds/rds.ko 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/rds/rds_rdma.ko 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/rds/rds_tcp.ko 2024-08-25T19:39:20.326 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_csum.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_gact.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_mirred.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_nat.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_pedit.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_police.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_simple.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_skbedit.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_vlan.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/cls_basic.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/cls_bpf.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/cls_flow.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/cls_fw.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/cls_route.ko 2024-08-25T19:39:20.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/cls_u32.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/em_cmp.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/em_ipset.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/em_meta.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/em_nbyte.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/em_text.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/em_u32.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_choke.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_codel.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_drr.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_fq.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_gred.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_hfsc.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_hhf.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_htb.ko 2024-08-25T19:39:20.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_ingress.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_mqprio.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_mqprio_lib.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_multiq.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_netem.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_pie.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_plug.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_prio.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_qfq.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_red.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_sfb.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_sfq.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_tbf.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_teql.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sctp 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sctp/sctp.ko 2024-08-25T19:39:20.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sctp/sctp_diag.ko 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc/auth_gss 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc/sunrpc.ko 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc/xprtrdma 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/tipc 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/tipc/tipc.ko 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/tipc/tipc_diag.ko 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/unix 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/unix/unix_diag.ko 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/vmw_vsock 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-08-25T19:39:20.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/vmw_vsock/vsock.ko 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/vmw_vsock/vsock_diag.ko 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/vmw_vsock/vsock_loopback.ko 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/xfrm 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/xfrm/xfrm_ipcomp.ko 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/security 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/security/keys 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/security/keys/encrypted-keys 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/security/keys/trusted-keys 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/security/keys/trusted-keys/trusted.ko 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.alias 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.alias.bin 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.builtin 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.builtin.alias.bin 2024-08-25T19:39:20.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.builtin.bin 2024-08-25T19:39:20.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.builtin.modinfo 2024-08-25T19:39:20.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.dep 2024-08-25T19:39:20.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.dep.bin 2024-08-25T19:39:20.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.devname 2024-08-25T19:39:20.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.order 2024-08-25T19:39:20.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.softdep 2024-08-25T19:39:20.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.symbols 2024-08-25T19:39:20.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.symbols.bin 2024-08-25T19:39:20.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.weakdep 2024-08-25T19:39:20.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/vmlinuz 2024-08-25T19:39:20.336 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:20.336 DEBUG:teuthology.orchestra.run.smithi119:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-08-25T19:39:20.359 INFO:teuthology.orchestra.run.smithi119.stdout:postinstall scriptlet (using /bin/sh): 2024-08-25T19:39:20.359 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-25T19:39:20.359 INFO:teuthology.orchestra.run.smithi119.stdout: /usr/bin/kernel-install add 6.11.0-rc3-g8618c856c9a6 /lib/modules/6.11.0-rc3-g8618c856c9a6/vmlinuz 2024-08-25T19:39:20.359 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout:for file in vmlinuz System.map config; do 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc3-g8618c856c9a6/${file}" "/boot/${file}-6.11.0-rc3-g8618c856c9a6"; then 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: cp "/lib/modules/6.11.0-rc3-g8618c856c9a6/${file}" "/boot/${file}-6.11.0-rc3-g8618c856c9a6" 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: fi 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout:done 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -d "/lib/modules/6.11.0-rc3-g8618c856c9a6/dtb" ] && \ 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: ! diff -rq "/lib/modules/6.11.0-rc3-g8618c856c9a6/dtb" "/boot/dtb-6.11.0-rc3-g8618c856c9a6" >/dev/null 2>&1; then 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: rm -rf "/boot/dtb-6.11.0-rc3-g8618c856c9a6" 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: cp -r "/lib/modules/6.11.0-rc3-g8618c856c9a6/dtb" "/boot/dtb-6.11.0-rc3-g8618c856c9a6" 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout:if [ ! -e "/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.dep" ]; then 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout: /usr/sbin/depmod 6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout:preuninstall scriptlet (using /bin/sh): 2024-08-25T19:39:20.360 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-25T19:39:20.361 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-install remove 6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:20.361 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2024-08-25T19:39:20.361 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2024-08-25T19:39:20.443 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2024-08-25T19:39:20.443 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2024-08-25T19:39:20.443 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.06 2024-08-25T19:39:20.443 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 82.el9 2024-08-25T19:39:20.443 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2024-08-25T19:39:20.443 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Wed 21 Aug 2024 09:12:29 PM UTC 2024-08-25T19:39:20.443 INFO:teuthology.orchestra.run.smithi119.stdout:Group : Unspecified 2024-08-25T19:39:20.443 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 8274670 2024-08-25T19:39:20.443 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : builder@centos.org 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : CentOS 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2024-08-25T19:39:20.444 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2024-08-25T19:39:20.445 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:39:20.445 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2024-08-25T19:39:20.445 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-08-25T19:39:20.446 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:20.446 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-08-25T19:39:21.022 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2024-08-25T19:39:22.369 INFO:teuthology.orchestra.run.smithi119.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-08-25T19:39:22.407 INFO:teuthology.orchestra.run.smithi119.stderr:done 2024-08-25T19:39:22.415 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-08-25T19:39:22.430 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.zggA6p2zMU 2024-08-25T19:39:22.431 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zggA6p2zMU 2024-08-25T19:39:22.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.zggA6p2zMU 2024-08-25T19:39:22.655 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.zggA6p2zMU is 6KB 2024-08-25T19:39:22.703 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.zggA6p2zMU 2024-08-25T19:39:22.717 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2024-08-25T19:39:22.780 INFO:teuthology.orchestra.run.smithi119.stdout:7acb1e34f9714b649e59f4e98f1746e5-0-rescue.conf 2024-08-25T19:39:22.780 INFO:teuthology.orchestra.run.smithi119.stdout:7acb1e34f9714b649e59f4e98f1746e5-5.14.0-239.el9.x86_64.conf 2024-08-25T19:39:22.780 INFO:teuthology.orchestra.run.smithi119.stdout:7acb1e34f9714b649e59f4e98f1746e5-5.14.0-496.el9.x86_64.conf 2024-08-25T19:39:22.780 INFO:teuthology.orchestra.run.smithi119.stdout:7acb1e34f9714b649e59f4e98f1746e5-6.11.0-rc3-g8618c856c9a6.conf 2024-08-25T19:39:22.781 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default 7acb1e34f9714b649e59f4e98f1746e5-6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:22.912 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2024-08-25T19:39:29.685 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi121.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-08-25T19:39:29.686 DEBUG:teuthology.orchestra.run.smithi121:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-08-25T19:39:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:/boot/System.map-6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:/boot/config-6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:/boot/vmlinuz-6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/System.map 2024-08-25T19:39:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/config 2024-08-25T19:39:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel 2024-08-25T19:39:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch 2024-08-25T19:39:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86 2024-08-25T19:39:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto 2024-08-25T19:39:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-08-25T19:39:29.760 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/crc32c-intel.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/serpent-avx2.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-08-25T19:39:29.761 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-08-25T19:39:29.762 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-08-25T19:39:29.762 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-08-25T19:39:29.762 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-08-25T19:39:29.762 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-08-25T19:39:29.762 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kernel 2024-08-25T19:39:29.762 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kernel/cpu 2024-08-25T19:39:29.762 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kernel/cpu/mce 2024-08-25T19:39:29.762 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-08-25T19:39:29.762 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kvm 2024-08-25T19:39:29.762 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kvm/kvm-amd.ko 2024-08-25T19:39:29.762 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kvm/kvm-intel.ko 2024-08-25T19:39:29.762 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/arch/x86/kvm/kvm.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/ansi_cprng.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/anubis.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/arc4.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx/async_memcpy.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx/async_pq.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx/async_raid6_recov.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx/async_tx.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx/async_xor.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/async_tx/raid6test.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/authenc.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/authencesn.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/blake2b_generic.ko 2024-08-25T19:39:29.763 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/blowfish_common.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/blowfish_generic.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/camellia_generic.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/cast5_generic.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/cast6_generic.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/cast_common.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/ccm.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/cmac.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/crc32_generic.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/crypto_engine.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/crypto_user.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/cts.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/des_generic.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/dh_generic.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/ecc.ko 2024-08-25T19:39:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/ecdh_generic.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/echainiv.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/essiv.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/fcrypt.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/khazad.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/lz4.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/lz4hc.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/md4.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/michael_mic.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/pcbc.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/pcrypt.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/rmd160.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/seed.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/serpent_generic.ko 2024-08-25T19:39:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/tcrypt.ko 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/tea.ko 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/twofish_common.ko 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/twofish_generic.ko 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/vmac.ko 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/wp512.ko 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/xcbc.ko 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/xor.ko 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/crypto/xxhash_generic.ko 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/acpi_ipmi.ko 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/acpi_pad.ko 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/ec_sys.ko 2024-08-25T19:39:29.766 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/platform_profile.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/sbs.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/sbshc.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/acpi/video.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/acard-ahci.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/ahci_platform.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/ata_generic.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/libahci_platform.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_acpi.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_ali.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_amd.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_artop.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_atiixp.ko 2024-08-25T19:39:29.767 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_atp867x.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_cmd640.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_cmd64x.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_cypress.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_efar.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_hpt366.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_hpt37x.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_hpt3x2n.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_hpt3x3.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_it8213.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_it821x.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_jmicron.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_marvell.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_mpiix.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_netcell.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_ninja32.ko 2024-08-25T19:39:29.768 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_ns87410.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_ns87415.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_oldpiix.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_opti.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_optidma.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_pcmcia.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_pdc2027x.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_piccolo.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_rdc.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_sch.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_serverworks.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_sil680.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_sis.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_sl82c105.ko 2024-08-25T19:39:29.769 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_triflex.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pata_via.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/pdc_adma.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_inic162x.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_mv.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_nv.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_promise.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_qstor.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_sil.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_sil24.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_sis.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_svw.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_sx4.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_uli.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_via.ko 2024-08-25T19:39:29.770 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ata/sata_vsc.ko 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/auxdisplay 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/auxdisplay/ks0108.ko 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/base 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/base/regmap 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/base/regmap/regmap-i2c.ko 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/base/regmap/regmap-mmio.ko 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/bcma 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/bcma/bcma.ko 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/aoe 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/aoe/aoe.ko 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/brd.ko 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/drbd 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/drbd/drbd.ko 2024-08-25T19:39:29.771 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/floppy.ko 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/loop.ko 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/mtip32xx 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/nbd.ko 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/null_blk 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/null_blk/null_blk.ko 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/pktcdvd.ko 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/rbd.ko 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/virtio_blk.ko 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/xen-blkback 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/xen-blkfront.ko 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/zram 2024-08-25T19:39:29.772 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/block/zram/zram.ko 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hangcheck-timer.ko 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hw_random 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hw_random/amd-rng.ko 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hw_random/intel-rng.ko 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hw_random/via-rng.ko 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/hw_random/virtio-rng.ko 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi/ipmi_si.ko 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-08-25T19:39:29.773 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-08-25T19:39:29.774 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/lp.ko 2024-08-25T19:39:29.774 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/mwave 2024-08-25T19:39:29.774 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/mwave/mwave.ko 2024-08-25T19:39:29.774 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/ppdev.ko 2024-08-25T19:39:29.774 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tlclk.ko 2024-08-25T19:39:29.774 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm 2024-08-25T19:39:29.774 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm/tpm.ko 2024-08-25T19:39:29.774 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm/tpm_atmel.ko 2024-08-25T19:39:29.774 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm/tpm_infineon.ko 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm/tpm_nsc.ko 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm/tpm_tis.ko 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/uv_mmtimer.ko 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/char/virtio_console.ko 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq/p4-clockmod.ko 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq/powernow-k8.ko 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/cpufreq/speedstep-lib.ko 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto 2024-08-25T19:39:29.775 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/ccp 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/ccp/ccp.ko 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/intel 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/intel/qat 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/intel/qat/qat_common 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/padlock-aes.ko 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/padlock-sha.ko 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/virtio 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dca 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dca/dca.ko 2024-08-25T19:39:29.776 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/devfreq 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dma 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dma/dw 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dma/dw/dw_dmac.ko 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dma/ioat 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/dma/ioat/ioatdma.ko 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/amd64_edac.ko 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/e752x_edac.ko 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/edac_mce_amd.ko 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i3000_edac.ko 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i3200_edac.ko 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i5100_edac.ko 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i5400_edac.ko 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i7300_edac.ko 2024-08-25T19:39:29.777 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i7core_edac.ko 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/i82975x_edac.ko 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/ie31200_edac.ko 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/sb_edac.ko 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/edac/x38_edac.ko 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/firmware 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/firmware/edd.ko 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/firmware/iscsi_ibft.ko 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/gpio 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/gpio/gpio-ich.ko 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/gpio/gpio-viperboard.ko 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-appleir.ko 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-aureal.ko 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-axff.ko 2024-08-25T19:39:29.778 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-dr.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-elecom.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-elo.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-emsff.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-gaff.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-gt683r.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-gyration.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-holtek-kbd.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-holtek-mouse.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-holtekff.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-hyperv.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-icade.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-keytouch.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-kye.ko 2024-08-25T19:39:29.779 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-lcpower.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-led.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-lenovo.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-logitech-dj.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-multitouch.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-ortek.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-penmount.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-petalynx.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-picolcd.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-pl.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-plantronics.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-primax.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-rmi.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-arvo.ko 2024-08-25T19:39:29.780 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-common.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-isku.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-kone.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-konepure.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-lua.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-pyra.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-ryos.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat-savu.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-roccat.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-saitek.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-samsung.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-sensor-hub.ko 2024-08-25T19:39:29.781 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-sjoy.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-sony.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-speedlink.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-steelseries.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-sunplus.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-thrustmaster.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-tivo.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-tmff.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-topseed.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-twinhan.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-uclogic.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-waltop.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-wiimote.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-xinmo.ko 2024-08-25T19:39:29.782 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-zpff.ko 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/hid-zydacron.ko 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/uhid.ko 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hid/wacom.ko 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hv 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hv/hv_balloon.ko 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hv/hv_utils.ko 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hv/hv_vmbus.ko 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/abituguru.ko 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/abituguru3.ko 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/acpi_power_meter.ko 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ad7414.ko 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ad7418.ko 2024-08-25T19:39:29.783 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adc128d818.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adm1025.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adm1026.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adm1029.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adm1031.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adm9240.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ads7828.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adt7410.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adt7411.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adt7462.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adt7470.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adt7475.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/adt7x10.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/amc6821.ko 2024-08-25T19:39:29.784 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/applesmc.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/asb100.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/asc7621.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/asus_atk0110.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/atxp1.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/coretemp.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/dme1737.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ds1621.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ds620.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/emc1403.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/emc6w201.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/f71805f.ko 2024-08-25T19:39:29.785 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/f71882fg.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/f75375s.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/fam15h_power.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/fschmd.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/g760a.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/g762.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/gl518sm.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/gl520sm.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/hwmon-vid.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/i5500_temp.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/i5k_amb.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ibmaem.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ibmpex.ko 2024-08-25T19:39:29.786 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ina209.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ina2xx.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/it87.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/k10temp.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/k8temp.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lineage-pem.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm63.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm73.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm75.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm77.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm78.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm80.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm83.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm85.ko 2024-08-25T19:39:29.787 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm87.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm90.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm92.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm93.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm95234.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm95241.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/lm95245.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc2945.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc4151.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc4215.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc4222.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc4245.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc4260.ko 2024-08-25T19:39:29.788 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/ltc4261.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max16065.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max1619.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max1668.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max197.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max6639.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max6650.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/max6697.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/mcp3021.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/nct6683.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/nct6775-core.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/nct6775.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/nct7802.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pc87360.ko 2024-08-25T19:39:29.789 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pc87427.ko 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pcf8591.ko 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/max16064.ko 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/max34440.ko 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/max8688.ko 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-08-25T19:39:29.790 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/powr1220.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/sch5627.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/sch5636.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/sch56xx-common.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/sht15.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/sht21.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/shtc1.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/sis5595.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/smsc47b397.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/smsc47m1.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/smsc47m192.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/thmc50.ko 2024-08-25T19:39:29.791 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/tmp102.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/tmp103.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/tmp401.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/tmp421.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/via-cputemp.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/via686a.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/vt1211.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/vt8231.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83627ehf.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83627hf.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83781d.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83791d.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83792d.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83793.ko 2024-08-25T19:39:29.792 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83795.ko 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83l785ts.ko 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/hwmon/w83l786ng.ko 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/algos 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-08-25T19:39:29.793 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-i801.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-isch.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-parport.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-08-25T19:39:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-via.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/i2c-dev.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/i2c-mux.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/i2c-smbus.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/i2c/i2c-stub.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/ib_cm.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/ib_core.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/ib_umad.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/iw_cm.ko 2024-08-25T19:39:29.795 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/rdma_cm.ko 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/cxgb4 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/mlx4 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/mlx5 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/mthca 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/ipoib 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-08-25T19:39:29.796 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/iser 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/isert 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/srp 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/srpt 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/gameport 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/gameport/fm801-gp.ko 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/gameport/gameport.ko 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/joydev.ko 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/keyboard 2024-08-25T19:39:29.797 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/keyboard/gpio_keys.ko 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/appletouch.ko 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/bcm5974.ko 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/cyapatp.ko 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/elan_i2c.ko 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/sermouse.ko 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/synaptics_usb.ko 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/mouse/vsxxxaa.ko 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/rmi4 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/rmi4/rmi_core.ko 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/serio 2024-08-25T19:39:29.798 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/serio/altera_ps2.ko 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/serio/arc_ps2.ko 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/serio/serio_raw.ko 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/input/sparse-keymap.ko 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/leds-blinkm.ko 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/leds-lm3530.ko 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/leds-lp3944.ko 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/leds-ss4200.ko 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-08-25T19:39:29.799 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/bcache 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/bcache/bcache.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-bio-prison.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-cache-smq.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-cache.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-crypt.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-delay.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-flakey.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-log-userspace.ko 2024-08-25T19:39:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-multipath.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-queue-length.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-raid.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-round-robin.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-service-time.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-switch.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-thin-pool.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/dm-verity.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/persistent-data 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/raid0.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/raid1.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/raid10.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/md/raid456.ko 2024-08-25T19:39:29.801 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptbase.ko 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptctl.ko 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptfc.ko 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptlan.ko 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptsas.ko 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptscsih.ko 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/message/fusion/mptspi.ko 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/lpc_ich.ko 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/lpc_sch.ko 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/mfd-core.ko 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/sm501.ko 2024-08-25T19:39:29.802 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/viperboard.ko 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/vx855.ko 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/mfd/wl1273-core.ko 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/altera-stapl 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/apds9802als.ko 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/apds990x.ko 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/bh1770glc.ko 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/cb710 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/cb710/cb710.ko 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/eeprom 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/eeprom/at24.ko 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-08-25T19:39:29.803 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/eeprom/max6875.ko 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/enclosure.ko 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/hpilo.ko 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/isl29003.ko 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/isl29020.ko 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/lis3lv02d 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/mei 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/mei/mei-me.ko 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/mei/mei-txe.ko 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/mei/mei.ko 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/sgi-gru 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/sgi-gru/gru.ko 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/sgi-xp 2024-08-25T19:39:29.804 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/sgi-xp/xp.ko 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/sgi-xp/xpc.ko 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/tifm_7xx1.ko 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/tifm_core.ko 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/tsl2550.ko 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/vmw_balloon.ko 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/vmw_vmci 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/bonding 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/bonding/bonding.ko 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa/b53 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa/b53/b53_common.ko 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa/bcm-sf2.ko 2024-08-25T19:39:29.805 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa/mv88e6060.ko 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa/mv88e6xxx 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/dummy.ko 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/eql.ko 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/3com 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/8390 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/8390/8390.ko 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-08-25T19:39:29.806 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/adaptec 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/agere 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/agere/et131x.ko 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/alteon 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/altera 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/amd 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-08-25T19:39:29.807 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros 2024-08-25T19:39:29.808 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/alx 2024-08-25T19:39:29.808 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-08-25T19:39:29.808 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atl1c 2024-08-25T19:39:29.808 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-08-25T19:39:29.808 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atl1e 2024-08-25T19:39:29.808 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-08-25T19:39:29.808 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atlx 2024-08-25T19:39:29.808 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-08-25T19:39:29.808 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-08-25T19:39:29.808 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom 2024-08-25T19:39:29.808 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-08-25T19:39:29.808 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/genet 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/brocade 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/brocade/bna 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-08-25T19:39:29.809 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/cisco 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/cisco/enic 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-08-25T19:39:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dlink 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/dnet.ko 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/emulex 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/emulex/benet 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/ethoc.ko 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/fealnx.ko 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/e100.ko 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/e1000 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-08-25T19:39:29.811 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/e1000e 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/fm10k 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/i40e 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/iavf 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/igb 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/igbvf 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/ixgbe 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/ixgbevf 2024-08-25T19:39:29.812 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/libeth 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/libie 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/jme.ko 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/marvell 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/marvell/skge.ko 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox/mlx4 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-08-25T19:39:29.813 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox/mlx5 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/micrel 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/myricom 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/myricom/myri10ge 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/natsemi 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/neterion 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-08-25T19:39:29.814 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/nvidia 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/packetengines 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/qlogic 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/qlogic/netxen 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/rdc 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-08-25T19:39:29.815 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/realtek 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/realtek/atp.ko 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/rocker 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sfc 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/silan 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sis 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sis/sis190.ko 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sis/sis900.ko 2024-08-25T19:39:29.816 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/smsc 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/stmicro 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/stmicro/stmmac 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sun 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sun/cassini.ko 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sun/niu.ko 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sun/sungem.ko 2024-08-25T19:39:29.817 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-08-25T19:39:29.818 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/tehuti 2024-08-25T19:39:29.818 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-08-25T19:39:29.818 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/ti 2024-08-25T19:39:29.818 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/ti/tlan.ko 2024-08-25T19:39:29.818 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/via 2024-08-25T19:39:29.818 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-08-25T19:39:29.818 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-08-25T19:39:29.818 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/wiznet 2024-08-25T19:39:29.818 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-08-25T19:39:29.818 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-08-25T19:39:29.818 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/xircom 2024-08-25T19:39:29.818 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/geneve.ko 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/hyperv 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ieee802154 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ieee802154/fakelb.ko 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ifb.ko 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ipvlan 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ipvlan/ipvlan.ko 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/macvlan.ko 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/macvtap.ko 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/mdio 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/mdio.ko 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-08-25T19:39:29.819 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/mdio/mdio-regmap.ko 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/mii.ko 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/net_failover.ko 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/netconsole.ko 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/nlmon.ko 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/pcs 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/pcs/pcs-lynx.ko 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/amd.ko 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-08-25T19:39:29.820 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/bcm7xxx.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/bcm87xx.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/broadcom.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/cicada.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/davicom.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/dp83640.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/et1011c.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/icplus.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/lxt.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/marvell.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/micrel.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/national.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/phylink.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/qsemi.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/realtek.ko 2024-08-25T19:39:29.821 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/smsc.ko 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/ste10Xp.ko 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/phy/vitesse.ko 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/bsd_comp.ko 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/ppp_async.ko 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/ppp_deflate.ko 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/ppp_generic.ko 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/ppp_mppe.ko 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/ppp_synctty.ko 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/pppoe.ko 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/pppox.ko 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/ppp/pptp.ko 2024-08-25T19:39:29.822 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/slip 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/slip/slhc.ko 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/slip/slip.ko 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/sungem_phy.ko 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/tap.ko 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team/team.ko 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team/team_mode_activebackup.ko 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team/team_mode_broadcast.ko 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team/team_mode_random.ko 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/tun.ko 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/veth.ko 2024-08-25T19:39:29.823 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/virtio_net.ko 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/vmxnet3 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/vxlan 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/vxlan/vxlan.ko 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/xen-netback 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/xen-netback/xen-netback.ko 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/net/xen-netfront.ko 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ntb 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ntb/ntb.ko 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/nvme 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/nvme/host 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/nvme/host/nvme-core.ko 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/nvme/host/nvme.ko 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/parport 2024-08-25T19:39:29.824 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/parport/parport.ko 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/parport/parport_cs.ko 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/parport/parport_pc.ko 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/parport/parport_serial.ko 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pci 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pci/hotplug 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pci/pcie 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pci/pcie/aer_inject.ko 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pci/xen-pcifront.ko 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pcmcia 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pcmcia/i82092.ko 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pcmcia/pd6729.ko 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pcmcia/yenta_socket.ko 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pinctrl 2024-08-25T19:39:29.825 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pinctrl/intel 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/acer-wmi.ko 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/acerhdf.ko 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/apple-gmux.ko 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/asus-laptop.ko 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/asus-wmi.ko 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/classmate-laptop.ko 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-08-25T19:39:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/hdaps.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/intel 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/intel_ips.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/msi-wmi.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/mxm-wmi.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/samsung-laptop.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/samsung-q10.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/topstar-laptop.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/toshiba_haps.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/wmi-bmof.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/platform/x86/wmi.ko 2024-08-25T19:39:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/powercap 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/powercap/intel_rapl_common.ko 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/powercap/intel_rapl_msr.ko 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pps 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pps/clients 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pps/clients/pps-gpio.ko 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pps/clients/pps-ldisc.ko 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pps/clients/pps_parport.ko 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/pps/pps_core.ko 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ptp 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ptp/ptp.ko 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ptp/ptp_kvm.ko 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ras 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ras/amd 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ras/amd/atl 2024-08-25T19:39:29.828 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ras/amd/fmpm.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-bq32k.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1286.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1307.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1374.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1511.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1553.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1672.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds1742.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds2404.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-ds3232.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-em3027.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-fm3130.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-isl12022.ko 2024-08-25T19:39:29.829 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-isl1208.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-m41t80.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-m48t35.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-m48t59.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-max6900.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-msm6242.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-pcf2127.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-pcf85063.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-pcf8523.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-pcf8563.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-pcf8583.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-rp5c01.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-rs5c372.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-08-25T19:39:29.830 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-rx8025.ko 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-rx8581.ko 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/rtc/rtc-x1205.ko 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/3w-9xxx.ko 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/3w-sas.ko 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/3w-xxxx.ko 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/BusLogic.ko 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/a100u2w.ko 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/aacraid 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/aacraid/aacraid.ko 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/advansys.ko 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/aic7xxx 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-08-25T19:39:29.831 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/am53c974.ko 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/arcmsr 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/atp870u.ko 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/be2iscsi 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/bfa 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/bfa/bfa.ko 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/bnx2fc 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/bnx2i 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/ch.ko 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/csiostor 2024-08-25T19:39:29.832 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/csiostor/csiostor.ko 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/cxgbi 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/cxgbi/cxgb3i 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/cxgbi/cxgb4i 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/dc395x.ko 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/device_handler 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/dmx3191d.ko 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/esas2r 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/esas2r/esas2r.ko 2024-08-25T19:39:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/esp_scsi.ko 2024-08-25T19:39:29.834 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/fcoe 2024-08-25T19:39:29.834 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/fcoe/fcoe.ko 2024-08-25T19:39:29.834 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-08-25T19:39:29.834 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/fnic 2024-08-25T19:39:29.834 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/fnic/fnic.ko 2024-08-25T19:39:29.834 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/hpsa.ko 2024-08-25T19:39:29.834 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/hptiop.ko 2024-08-25T19:39:29.834 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/hv_storvsc.ko 2024-08-25T19:39:29.834 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/initio.ko 2024-08-25T19:39:29.834 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/ipr.ko 2024-08-25T19:39:29.834 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/ips.ko 2024-08-25T19:39:29.834 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/isci 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/isci/isci.ko 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/iscsi_tcp.ko 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/libfc 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/libfc/libfc.ko 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/libiscsi.ko 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/libiscsi_tcp.ko 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/libsas 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/libsas/libsas.ko 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/lpfc 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/lpfc/lpfc.ko 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/megaraid 2024-08-25T19:39:29.835 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/megaraid.ko 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/mpt3sas 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/mvsas 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/mvsas/mvsas.ko 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/mvumi.ko 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/pm8001 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/pmcraid.ko 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/qla1280.ko 2024-08-25T19:39:29.836 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/qla2xxx 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/qla4xxx 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/raid_class.ko 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/scsi_debug.ko 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/scsi_transport_fc.ko 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/scsi_transport_sas.ko 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/scsi_transport_spi.ko 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/scsi_transport_srp.ko 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/ses.ko 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/st.ko 2024-08-25T19:39:29.837 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/stex.ko 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/sym53c8xx_2 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/virtio_scsi.ko 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/vmw_pvscsi.ko 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/scsi/wd719x.ko 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ssb 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ssb/ssb.ko 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/iscsi 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/loopback 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/loopback/tcm_loop.ko 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/target_core_file.ko 2024-08-25T19:39:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/target_core_iblock.ko 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/target_core_mod.ko 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/target_core_pscsi.ko 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/target_core_user.ko 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/tcm_fc 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-08-25T19:39:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty 2024-08-25T19:39:29.840 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/ipwireless 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/n_gsm.ko 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/n_hdlc.ko 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/nozomi.ko 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/serial 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/serial/8250 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/serial/arc_uart.ko 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/serial/jsm 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/tty/serial/jsm/jsm.ko 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ufs 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ufs/core 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ufs/core/ufshcd-core.ko 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ufs/host 2024-08-25T19:39:29.841 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/uio 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/uio/uio.ko 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/uio/uio_aec.ko 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/uio/uio_cif.ko 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/uio/uio_pci_generic.ko 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/uio/uio_sercos3.ko 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/class 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/class/cdc-acm.ko 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/class/cdc-wdm.ko 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/class/usblp.ko 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/class/usbtmc.ko 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/host 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/host/sl811-hcd.ko 2024-08-25T19:39:29.842 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/image 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/image/mdc800.ko 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/image/microtek.ko 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/adutux.ko 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/appledisplay.ko 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/emi26.ko 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/emi62.ko 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/ezusb.ko 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/idmouse.ko 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/iowarrior.ko 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/isight_firmware.ko 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/ldusb.ko 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/legousbtower.ko 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/sisusbvga 2024-08-25T19:39:29.843 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/trancevibrator.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/usb3503.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/usblcd.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/usbsevseg.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/uss720.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/misc/yurex.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/phy 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/phy/phy-generic.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/aircable.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ark3116.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/belkin_sa.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ch341.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/cp210x.ko 2024-08-25T19:39:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/cyberjack.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/cypress_m8.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/digi_acceleport.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/empeg.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ftdi_sio.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/garmin_gps.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/io_edgeport.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/io_ti.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ipaq.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ipw.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ir-usb.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/keyspan.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/keyspan_pda.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/kl5kusb105.ko 2024-08-25T19:39:29.845 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/kobil_sct.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/mct_u232.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/mos7720.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/mos7840.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/navman.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/omninet.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/opticon.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/option.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/oti6858.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/pl2303.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/qcaux.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/qcserial.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/quatech2.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/safe_serial.ko 2024-08-25T19:39:29.846 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/sierra.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/spcp8x5.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ssu100.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/symbolserial.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/usb_debug.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/usb_wwan.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/visor.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/whiteheat.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/serial/xsens_mt.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/uas.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-alauda.ko 2024-08-25T19:39:29.847 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-cypress.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-datafab.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-freecom.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-isd200.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-karma.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-onetouch.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-realtek.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-sddr09.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-sddr55.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/ums-usbat.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/storage/usb-storage.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/usbip 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/usbip/usbip-core.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/usbip/usbip-host.ko 2024-08-25T19:39:29.848 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vfio 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vfio/pci 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vfio/pci/vfio-pci.ko 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vfio/vfio.ko 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vhost 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vhost/vhost.ko 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vhost/vhost_iotlb.ko 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vhost/vhost_net.ko 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/vhost/vhost_scsi.ko 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/video 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/video/backlight 2024-08-25T19:39:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/video/backlight/apple_bl.ko 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/video/backlight/lcd.ko 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/video/backlight/platform_lcd.ko 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio.ko 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio_balloon.ko 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio_mmio.ko 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio_pci.ko 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/virtio/virtio_ring.ko 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/masters 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/masters/ds2482.ko 2024-08-25T19:39:29.850 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/masters/ds2490.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_smem.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/slaves/w1_therm.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/w1/wire.ko 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog 2024-08-25T19:39:29.851 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/alim1535_wdt.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/alim7101_wdt.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/f71808e_wdt.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/hpwdt.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/i6300esb.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/iTCO_wdt.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/ib700wdt.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/ibmasr.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/it8712f_wdt.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/it87_wdt.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/machzwd.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/nv_tco.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/pcwd_pci.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/pcwd_usb.ko 2024-08-25T19:39:29.852 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/sch311x_wdt.ko 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/softdog.ko 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/sp5100_tco.ko 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/via_wdt.ko 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/w83877f_wdt.ko 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/w83977f_wdt.ko 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/wdt_pci.ko 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/watchdog/xen_wdt.ko 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-acpi-processor.ko 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-evtchn.ko 2024-08-25T19:39:29.853 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-gntalloc.ko 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-gntdev.ko 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-pciback 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xen-privcmd.ko 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xenfs 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/drivers/xen/xenfs/xenfs.ko 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/9p 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/9p/9p.ko 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/affs 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/affs/affs.ko 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/befs 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/befs/befs.ko 2024-08-25T19:39:29.854 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/binfmt_misc.ko 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/btrfs 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/btrfs/btrfs.ko 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/cachefiles 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/cachefiles/cachefiles.ko 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ceph 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ceph/ceph.ko 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/coda 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/coda/coda.ko 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/cramfs 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/cramfs/cramfs.ko 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/dlm 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/dlm/dlm.ko 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ecryptfs 2024-08-25T19:39:29.855 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ecryptfs/ecryptfs.ko 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/f2fs 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/f2fs/f2fs.ko 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fat 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fat/fat.ko 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fat/msdos.ko 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fat/vfat.ko 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fuse 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fuse/cuse.ko 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/fuse/fuse.ko 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/gfs2 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/gfs2/gfs2.ko 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/hfs 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/hfs/hfs.ko 2024-08-25T19:39:29.856 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/hfsplus 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/hfsplus/hfsplus.ko 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/isofs 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/isofs/isofs.ko 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/jfs 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/jfs/jfs.ko 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/lockd 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/lockd/lockd.ko 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/minix 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/minix/minix.ko 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/netfs 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/netfs/netfs.ko 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/blocklayout 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/filelayout 2024-08-25T19:39:29.857 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/flexfilelayout 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/nfs.ko 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/nfsv3.ko 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs/nfsv4.ko 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs_common 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs_common/grace.ko 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfs_common/nfs_acl.ko 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfsd 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nfsd/nfsd.ko 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nilfs2 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nilfs2/nilfs2.ko 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-celtic.ko 2024-08-25T19:39:29.858 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-centeuro.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-croatian.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-cyrillic.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-gaelic.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-greek.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-iceland.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-inuit.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-roman.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-romanian.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/mac-turkish.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp1250.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp1251.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp1255.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp737.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp775.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp850.ko 2024-08-25T19:39:29.859 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp852.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp855.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp857.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp860.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp861.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp862.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp863.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp864.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp865.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp866.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp869.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp874.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp932.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp936.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp949.ko 2024-08-25T19:39:29.860 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_cp950.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_euc-jp.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-1.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-13.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-14.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-15.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-2.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-3.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-4.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-5.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-6.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-7.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_iso8859-9.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_koi8-r.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_koi8-ru.ko 2024-08-25T19:39:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_koi8-u.ko 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_ucs2_utils.ko 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/nls/nls_utf8.ko 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/cluster 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/dlm 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/dlmfs 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/ocfs2.ko 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-08-25T19:39:29.862 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/overlayfs 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/overlayfs/overlay.ko 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/pstore 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/pstore/ramoops.ko 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/reiserfs 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/reiserfs/reiserfs.ko 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/romfs 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/romfs/romfs.ko 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/smb 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/smb/client 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/smb/client/cifs.ko 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/smb/common 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/smb/common/cifs_arc4.ko 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/smb/common/cifs_md4.ko 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/squashfs 2024-08-25T19:39:29.863 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/squashfs/squashfs.ko 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/sysv 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/sysv/sysv.ko 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/udf 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/udf/udf.ko 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ufs 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/ufs/ufs.ko 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/xfs 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/fs/xfs/xfs.ko 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/kernel 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/kernel/rcu 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/kernel/rcu/rcutorture.ko 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/kernel/torture.ko 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/kernel/trace 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib 2024-08-25T19:39:29.864 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/asn1_encoder.ko 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/crc-itu-t.ko 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/crc8.ko 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/crypto 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/crypto/libaescfb.ko 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/crypto/libarc4.ko 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/crypto/libdes.ko 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/libcrc32c.ko 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/lru_cache.ko 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/lz4 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/lz4/lz4_compress.ko 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/lz4/lz4hc_compress.ko 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/math 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/math/cordic.ko 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/raid6 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/raid6/raid6_pq.ko 2024-08-25T19:39:29.865 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/reed_solomon 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/reed_solomon/reed_solomon.ko 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/ts_bm.ko 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/ts_fsm.ko 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/ts_kmp.ko 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/zstd 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/lib/zstd/zstd_compress.ko 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/mm 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/mm/hwpoison-inject.ko 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/6lowpan.ko 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_dest.ko 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_fragment.ko 2024-08-25T19:39:29.866 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_hop.ko 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_ipv6.ko 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_mobility.ko 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_routing.ko 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/6lowpan/nhc_udp.ko 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/802 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/802/garp.ko 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/802/mrp.ko 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/802/p8022.ko 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/802/psnap.ko 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/802/stp.ko 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/8021q 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/8021q/8021q.ko 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/9p 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/9p/9pnet.ko 2024-08-25T19:39:29.867 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/9p/9pnet_fd.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/9p/9pnet_rdma.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/9p/9pnet_virtio.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/batman-adv 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/batman-adv/batman-adv.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/br_netfilter.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/bridge.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_among.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_arp.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_ip.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-08-25T19:39:29.868 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_limit.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_log.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_mark.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_snat.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_stp.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/ebtables.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-08-25T19:39:29.869 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ceph 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ceph/libceph.ko 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/core 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/core/failover.ko 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/core/pktgen.ko 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dccp 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dccp/dccp.ko 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dccp/dccp_diag.ko 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dccp/dccp_ipv4.ko 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dccp/dccp_ipv6.ko 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dns_resolver 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dns_resolver/dns_resolver.ko 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dsa 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dsa/dsa_core.ko 2024-08-25T19:39:29.870 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dsa/tag_brcm.ko 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dsa/tag_dsa.ko 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dsa/tag_none.ko 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/dsa/tag_trailer.ko 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ieee802154 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ieee802154/6lowpan 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ieee802154/ieee802154.ko 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ieee802154/ieee802154_socket.ko 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/ah4.ko 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/esp4.ko 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/fou.ko 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/gre.ko 2024-08-25T19:39:29.871 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/inet_diag.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/ip_gre.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/ip_tunnel.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/ip_vti.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/ipcomp.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/ipip.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/arp_tables.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-08-25T19:39:29.872 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/iptable_security.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_bic.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_dctcp.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_diag.ko 2024-08-25T19:39:29.873 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_highspeed.ko 2024-08-25T19:39:29.874 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_htcp.ko 2024-08-25T19:39:29.874 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_hybla.ko 2024-08-25T19:39:29.874 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_illinois.ko 2024-08-25T19:39:29.874 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_lp.ko 2024-08-25T19:39:29.874 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_scalable.ko 2024-08-25T19:39:29.874 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_vegas.ko 2024-08-25T19:39:29.874 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_veno.ko 2024-08-25T19:39:29.874 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_westwood.ko 2024-08-25T19:39:29.874 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tcp_yeah.ko 2024-08-25T19:39:29.874 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/tunnel4.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/udp_diag.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/udp_tunnel.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv4/xfrm4_tunnel.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/ah6.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/esp6.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/fou6.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/ip6_tunnel.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/ip6_vti.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/ipcomp6.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-08-25T19:39:29.875 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-08-25T19:39:29.876 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/sit.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/tunnel6.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/ipv6/xfrm6_tunnel.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/key 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/key/af_key.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_core.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_debugfs.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_eth.ko 2024-08-25T19:39:29.877 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_ip.ko 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_ip6.ko 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_netlink.ko 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/l2tp/l2tp_ppp.ko 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/llc 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/llc/llc.ko 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/mac802154 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/mac802154/mac802154.ko 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/mpls 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/mpls/mpls_gso.ko 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set.ko 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-08-25T19:39:29.878 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-08-25T19:39:29.879 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-08-25T19:39:29.880 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conncount.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_h323.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_irc.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_sane.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_sip.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-08-25T19:39:29.881 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_dup_netdev.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_flow_table.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_flow_table_inet.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_log_syslog.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_nat.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_nat_amanda.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_nat_ftp.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_nat_irc.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_nat_sip.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_nat_tftp.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_synproxy_core.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nf_tables.ko 2024-08-25T19:39:29.882 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nfnetlink.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nfnetlink_acct.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nfnetlink_log.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nfnetlink_osf.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nfnetlink_queue.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_chain_nat.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_compat.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_ct.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_dup_netdev.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_fib.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_fwd_netdev.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_hash.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_limit.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_log.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_masq.ko 2024-08-25T19:39:29.883 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_nat.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_numgen.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_queue.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_quota.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_redir.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_reject.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_reject_inet.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_socket.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/nft_tunnel.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_AUDIT.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_CHECKSUM.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_CLASSIFY.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_CT.ko 2024-08-25T19:39:29.884 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_DSCP.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_HL.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_HMARK.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_IDLETIMER.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_LED.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_LOG.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_MASQUERADE.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_NETMAP.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_NFLOG.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_NFQUEUE.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_RATEEST.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_REDIRECT.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_SECMARK.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_TCPMSS.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-08-25T19:39:29.885 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_TEE.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_TPROXY.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_TRACE.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_addrtype.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_bpf.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_cgroup.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_cluster.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_comment.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_connbytes.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_connlabel.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_connlimit.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_connmark.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_conntrack.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_cpu.ko 2024-08-25T19:39:29.886 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_dccp.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_devgroup.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_dscp.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_ecn.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_esp.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_hashlimit.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_helper.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_hl.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_ipcomp.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_iprange.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_ipvs.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_l2tp.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_length.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_limit.ko 2024-08-25T19:39:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_mac.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_mark.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_multiport.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_nat.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_nfacct.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_osf.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_owner.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_physdev.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_pkttype.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_policy.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_quota.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_rateest.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_realm.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_recent.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_sctp.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_set.ko 2024-08-25T19:39:29.888 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_socket.ko 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_state.ko 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_statistic.ko 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_string.ko 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_tcpmss.ko 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_time.ko 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netfilter/xt_u32.ko 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netlink 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/netlink/netlink_diag.ko 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/nsh 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/nsh/nsh.ko 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/openvswitch 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/openvswitch/openvswitch.ko 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/openvswitch/vport-geneve.ko 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/openvswitch/vport-gre.ko 2024-08-25T19:39:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/openvswitch/vport-vxlan.ko 2024-08-25T19:39:29.890 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/packet 2024-08-25T19:39:29.890 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/packet/af_packet_diag.ko 2024-08-25T19:39:29.890 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/rds 2024-08-25T19:39:29.890 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/rds/rds.ko 2024-08-25T19:39:29.890 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/rds/rds_rdma.ko 2024-08-25T19:39:29.890 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/rds/rds_tcp.ko 2024-08-25T19:39:29.890 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched 2024-08-25T19:39:29.890 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_csum.ko 2024-08-25T19:39:29.890 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_gact.ko 2024-08-25T19:39:29.890 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_mirred.ko 2024-08-25T19:39:29.890 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_nat.ko 2024-08-25T19:39:29.890 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_pedit.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_police.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_simple.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_skbedit.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/act_vlan.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/cls_basic.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/cls_bpf.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/cls_flow.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/cls_fw.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/cls_route.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/cls_u32.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/em_cmp.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/em_ipset.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/em_meta.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/em_nbyte.ko 2024-08-25T19:39:29.891 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/em_text.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/em_u32.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_choke.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_codel.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_drr.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_fq.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_gred.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_hfsc.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_hhf.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_htb.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_ingress.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_mqprio.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_mqprio_lib.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_multiq.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_netem.ko 2024-08-25T19:39:29.892 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_pie.ko 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_plug.ko 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_prio.ko 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_qfq.ko 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_red.ko 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_sfb.ko 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_sfq.ko 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_tbf.ko 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sched/sch_teql.ko 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sctp 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sctp/sctp.ko 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sctp/sctp_diag.ko 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc/auth_gss 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-08-25T19:39:29.893 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc/sunrpc.ko 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc/xprtrdma 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/tipc 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/tipc/tipc.ko 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/tipc/tipc_diag.ko 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/unix 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/unix/unix_diag.ko 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/vmw_vsock 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/vmw_vsock/vsock.ko 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/vmw_vsock/vsock_diag.ko 2024-08-25T19:39:29.894 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/vmw_vsock/vsock_loopback.ko 2024-08-25T19:39:29.895 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/xfrm 2024-08-25T19:39:29.895 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/net/xfrm/xfrm_ipcomp.ko 2024-08-25T19:39:29.895 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/security 2024-08-25T19:39:29.895 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/security/keys 2024-08-25T19:39:29.895 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/security/keys/encrypted-keys 2024-08-25T19:39:29.895 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-08-25T19:39:29.895 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/security/keys/trusted-keys 2024-08-25T19:39:29.895 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/kernel/security/keys/trusted-keys/trusted.ko 2024-08-25T19:39:29.895 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.alias 2024-08-25T19:39:29.895 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.alias.bin 2024-08-25T19:39:29.895 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.builtin 2024-08-25T19:39:29.896 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.builtin.alias.bin 2024-08-25T19:39:29.896 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.builtin.bin 2024-08-25T19:39:29.896 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.builtin.modinfo 2024-08-25T19:39:29.896 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.dep 2024-08-25T19:39:29.896 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.dep.bin 2024-08-25T19:39:29.896 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.devname 2024-08-25T19:39:29.896 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.order 2024-08-25T19:39:29.896 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.softdep 2024-08-25T19:39:29.896 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.symbols 2024-08-25T19:39:29.896 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.symbols.bin 2024-08-25T19:39:29.896 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.weakdep 2024-08-25T19:39:29.896 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.11.0-rc3-g8618c856c9a6/vmlinuz 2024-08-25T19:39:29.900 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:29.900 DEBUG:teuthology.orchestra.run.smithi121:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-08-25T19:39:29.922 INFO:teuthology.orchestra.run.smithi121.stdout:postinstall scriptlet (using /bin/sh): 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout: /usr/bin/kernel-install add 6.11.0-rc3-g8618c856c9a6 /lib/modules/6.11.0-rc3-g8618c856c9a6/vmlinuz 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout:fi 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout:for file in vmlinuz System.map config; do 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc3-g8618c856c9a6/${file}" "/boot/${file}-6.11.0-rc3-g8618c856c9a6"; then 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout: cp "/lib/modules/6.11.0-rc3-g8618c856c9a6/${file}" "/boot/${file}-6.11.0-rc3-g8618c856c9a6" 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout: fi 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout:done 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout:if [ -d "/lib/modules/6.11.0-rc3-g8618c856c9a6/dtb" ] && \ 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout: ! diff -rq "/lib/modules/6.11.0-rc3-g8618c856c9a6/dtb" "/boot/dtb-6.11.0-rc3-g8618c856c9a6" >/dev/null 2>&1; then 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout: rm -rf "/boot/dtb-6.11.0-rc3-g8618c856c9a6" 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout: cp -r "/lib/modules/6.11.0-rc3-g8618c856c9a6/dtb" "/boot/dtb-6.11.0-rc3-g8618c856c9a6" 2024-08-25T19:39:29.923 INFO:teuthology.orchestra.run.smithi121.stdout:fi 2024-08-25T19:39:29.924 INFO:teuthology.orchestra.run.smithi121.stdout:if [ ! -e "/lib/modules/6.11.0-rc3-g8618c856c9a6/modules.dep" ]; then 2024-08-25T19:39:29.924 INFO:teuthology.orchestra.run.smithi121.stdout: /usr/sbin/depmod 6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:29.924 INFO:teuthology.orchestra.run.smithi121.stdout:fi 2024-08-25T19:39:29.924 INFO:teuthology.orchestra.run.smithi121.stdout:preuninstall scriptlet (using /bin/sh): 2024-08-25T19:39:29.924 INFO:teuthology.orchestra.run.smithi121.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-25T19:39:29.924 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-install remove 6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:29.924 INFO:teuthology.orchestra.run.smithi121.stdout:fi 2024-08-25T19:39:29.925 DEBUG:teuthology.orchestra.run.smithi121:> sudo rpm -qi grub2-tools 2024-08-25T19:39:29.997 INFO:teuthology.orchestra.run.smithi121.stdout:Name : grub2-tools 2024-08-25T19:39:29.997 INFO:teuthology.orchestra.run.smithi121.stdout:Epoch : 1 2024-08-25T19:39:29.997 INFO:teuthology.orchestra.run.smithi121.stdout:Version : 2.06 2024-08-25T19:39:29.997 INFO:teuthology.orchestra.run.smithi121.stdout:Release : 82.el9 2024-08-25T19:39:29.997 INFO:teuthology.orchestra.run.smithi121.stdout:Architecture: x86_64 2024-08-25T19:39:29.997 INFO:teuthology.orchestra.run.smithi121.stdout:Install Date: Wed 21 Aug 2024 09:12:29 PM UTC 2024-08-25T19:39:29.997 INFO:teuthology.orchestra.run.smithi121.stdout:Group : Unspecified 2024-08-25T19:39:29.997 INFO:teuthology.orchestra.run.smithi121.stdout:Size : 8274670 2024-08-25T19:39:29.997 INFO:teuthology.orchestra.run.smithi121.stdout:License : GPLv3+ 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout:Packager : builder@centos.org 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout:Vendor : CentOS 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout:URL : http://www.gnu.org/software/grub/ 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout:Summary : Support tools for GRUB. 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout:Description : 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout:customizable bootloader with modular architecture. It supports a rich 2024-08-25T19:39:29.998 INFO:teuthology.orchestra.run.smithi121.stdout:variety of kernel formats, file systems, computer architectures and 2024-08-25T19:39:29.999 INFO:teuthology.orchestra.run.smithi121.stdout:hardware devices. 2024-08-25T19:39:29.999 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:39:29.999 INFO:teuthology.orchestra.run.smithi121.stdout:This subpackage provides tools for support of all platforms. 2024-08-25T19:39:29.999 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-08-25T19:39:30.000 INFO:teuthology.task.kernel:Updating grub on smithi121 to boot 6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:30.000 DEBUG:teuthology.orchestra.run.smithi121:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-08-25T19:39:30.563 INFO:teuthology.orchestra.run.smithi121.stderr:Generating grub configuration file ... 2024-08-25T19:39:31.919 INFO:teuthology.orchestra.run.smithi121.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-08-25T19:39:31.967 INFO:teuthology.orchestra.run.smithi121.stderr:done 2024-08-25T19:39:31.979 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-08-25T19:39:31.995 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.4V1vxf01xD 2024-08-25T19:39:31.995 DEBUG:teuthology.orchestra.run.smithi121:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4V1vxf01xD 2024-08-25T19:39:32.074 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.4V1vxf01xD 2024-08-25T19:39:32.211 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.4V1vxf01xD is 6KB 2024-08-25T19:39:32.259 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.4V1vxf01xD 2024-08-25T19:39:32.274 DEBUG:teuthology.orchestra.run.smithi121:> sudo /bin/ls /boot/loader/entries || true 2024-08-25T19:39:32.337 INFO:teuthology.orchestra.run.smithi121.stdout:7acb1e34f9714b649e59f4e98f1746e5-0-rescue.conf 2024-08-25T19:39:32.337 INFO:teuthology.orchestra.run.smithi121.stdout:7acb1e34f9714b649e59f4e98f1746e5-5.14.0-239.el9.x86_64.conf 2024-08-25T19:39:32.337 INFO:teuthology.orchestra.run.smithi121.stdout:7acb1e34f9714b649e59f4e98f1746e5-5.14.0-496.el9.x86_64.conf 2024-08-25T19:39:32.337 INFO:teuthology.orchestra.run.smithi121.stdout:7acb1e34f9714b649e59f4e98f1746e5-6.11.0-rc3-g8618c856c9a6.conf 2024-08-25T19:39:32.338 DEBUG:teuthology.orchestra.run.smithi121:> sudo grub2-set-default 7acb1e34f9714b649e59f4e98f1746e5-6.11.0-rc3-g8618c856c9a6 2024-08-25T19:39:32.468 DEBUG:teuthology.orchestra.run.smithi121:> sudo shutdown -r now 2024-08-25T19:39:52.933 INFO:teuthology.task.kernel:Checking client cluster1.mon.a for new kernel version... 2024-08-25T19:39:52.934 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-08-25T19:39:52.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-25T19:40:02.481 INFO:teuthology.task.kernel:Checking client cluster2.mon.a for new kernel version... 2024-08-25T19:40:02.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-08-25T19:40:02.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-25T19:40:11.323 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-08-25T19:40:20.326 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-08-25T19:40:20.327 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-25T19:40:21.051 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-08-25T19:40:23.387 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-08-25T19:40:30.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-08-25T19:40:30.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-25T19:40:33.115 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-08-25T19:40:35.388 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-08-25T19:40:35.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-25T19:40:45.119 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-08-25T19:40:45.119 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-25T19:41:07.165 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-08-25T19:41:07.714 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-08-25T19:41:07.714 INFO:teuthology.task.kernel:Checking kernel version of cluster1.mon.a, want "6.11.0-rc3-g8618c856c9a6"... 2024-08-25T19:41:07.715 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-08-25T19:41:07.749 INFO:teuthology.orchestra.run.smithi119.stdout:6.11.0-rc3-g8618c856c9a6 2024-08-25T19:41:07.749 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc3-g8618c856c9a6 vs 6.11.0-rc3-g8618c856c9a6 2024-08-25T19:41:07.749 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-25T19:41:07.750 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-08-25T19:41:08.751 INFO:teuthology.task.kernel:Enabling kdb on cluster1.mon.a... 2024-08-25T19:41:08.751 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-25T19:41:08.851 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2024-08-25T19:41:08.873 DEBUG:teuthology.parallel:result is None 2024-08-25T19:41:17.178 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-08-25T19:41:17.768 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-08-25T19:41:17.768 INFO:teuthology.task.kernel:Checking kernel version of cluster2.mon.a, want "6.11.0-rc3-g8618c856c9a6"... 2024-08-25T19:41:17.768 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2024-08-25T19:41:17.803 INFO:teuthology.orchestra.run.smithi121.stdout:6.11.0-rc3-g8618c856c9a6 2024-08-25T19:41:17.803 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc3-g8618c856c9a6 vs 6.11.0-rc3-g8618c856c9a6 2024-08-25T19:41:17.803 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-25T19:41:17.804 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-08-25T19:41:18.805 INFO:teuthology.task.kernel:Enabling kdb on cluster2.mon.a... 2024-08-25T19:41:18.805 DEBUG:teuthology.orchestra.run.smithi121:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-25T19:41:18.967 INFO:teuthology.orchestra.run.smithi121.stdout:ttyS1 2024-08-25T19:41:18.995 DEBUG:teuthology.parallel:result is None 2024-08-25T19:41:18.995 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-25T19:41:19.002 INFO:teuthology.task.internal:Creating test directory... 2024-08-25T19:41:19.002 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-25T19:41:19.007 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-25T19:41:19.043 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-25T19:41:19.049 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-25T19:41:19.060 INFO:teuthology.task.internal:Creating archive directory... 2024-08-25T19:41:19.060 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-25T19:41:19.089 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-25T19:41:19.247 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-25T19:41:19.254 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-25T19:41:19.254 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-25T19:41:19.258 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-25T19:41:19.315 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-25T19:41:19.328 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-25T19:41:19.338 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-25T19:41:19.351 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-25T19:41:19.355 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-25T19:41:19.361 INFO:teuthology.task.internal:Configuring sudo... 2024-08-25T19:41:19.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-25T19:41:19.365 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-25T19:41:19.422 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-25T19:41:19.432 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-25T19:41:19.432 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-25T19:41:19.436 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-25T19:41:19.472 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-25T19:41:19.570 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-25T19:41:19.660 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-25T19:41:19.723 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-25T19:41:19.792 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:41:19.792 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-25T19:41:19.846 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-25T19:41:19.911 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-25T19:41:19.984 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-25T19:41:20.049 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-25T19:41:20.125 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:41:20.126 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-25T19:41:20.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-08-25T19:41:20.184 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2024-08-25T19:41:20.263 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-25T19:41:20.309 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-25T19:41:20.664 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-25T19:41:20.679 INFO:teuthology.task.internal:Starting timer... 2024-08-25T19:41:20.679 INFO:teuthology.run_tasks:Running task pcp... 2024-08-25T19:41:20.690 INFO:teuthology.run_tasks:Running task selinux... 2024-08-25T19:41:20.699 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2024-08-25T19:41:20.784 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2024-08-25T19:41:20.788 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2024-08-25T19:41:20.876 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2024-08-25T19:41:20.880 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-25T19:41:20.880 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2024-08-25T19:41:20.921 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2024-08-25T19:41:20.922 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2024-08-25T19:41:20.972 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2024-08-25T19:41:20.973 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi121.front.sepia.ceph.com': 'permissive'} 2024-08-25T19:41:20.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-08-25T19:41:21.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-25T19:41:21.031 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-08-25T19:41:21.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-25T19:41:21.089 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-25T19:41:21.089 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2024-08-25T19:41:21.150 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2024-08-25T19:41:21.209 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-25T19:41:21.219 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-25T19:41:21.229 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-25T19:41:21.230 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi119.front.sepia.ceph.com,smithi121.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-25T19:43:27.667 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi121.front.sepia.ceph.com')] 2024-08-25T19:43:27.668 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-08-25T19:43:27.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-25T19:43:27.756 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-08-25T19:43:27.862 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-08-25T19:43:27.862 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-08-25T19:43:27.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-25T19:43:27.947 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-08-25T19:43:28.064 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-08-25T19:43:28.064 INFO:teuthology.run_tasks:Running task clock... 2024-08-25T19:43:28.074 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-25T19:43:28.074 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-25T19:43:28.075 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-25T19:43:28.079 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-25T19:43:28.079 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-25T19:43:28.136 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-25T19:43:28.165 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-25T19:43:28.165 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-25T19:43:28.194 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-25T19:43:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2024-08-25T19:43:28.239 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2024-08-25T19:43:28.246 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2024-08-25T19:43:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-25T19:43:28.270 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2024-08-25T19:43:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-25T19:43:28.299 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-25T19:43:28.329 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-25T19:43:28.396 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-08-25T19:43:28.402 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-25T19:43:28.402 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-08-25T19:43:28.402 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-25T19:43:28.402 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-25T19:43:28.402 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-25T19:43:28.402 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-25T19:43:28.424 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-08-25T19:43:28.431 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-25T19:43:28.431 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-08-25T19:43:28.431 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-25T19:43:28.431 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-25T19:43:28.431 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-25T19:43:28.431 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-25T19:43:28.432 INFO:teuthology.run_tasks:Running task install... 2024-08-25T19:43:28.440 DEBUG:teuthology.task.install:project ceph 2024-08-25T19:43:28.440 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12'}, 'extra_system_packages': ['pv']} 2024-08-25T19:43:28.440 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-mirror'], 'flavor': 'default', 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12', 'extra_system_packages': ['pv']} 2024-08-25T19:43:28.441 INFO:teuthology.task.install:Using flavor: default 2024-08-25T19:43:28.446 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-25T19:43:28.446 INFO:teuthology.task.install:extra packages: ['rbd-mirror'] 2024-08-25T19:43:28.446 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-mirror'], 'extra_system_packages': ['pv'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12', 'tag': None, 'wait_for_package': False} 2024-08-25T19:43:28.447 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=db4959e44f0252a53b4d8688fbf88b3dda70ea12 2024-08-25T19:43:28.449 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-mirror'], 'extra_system_packages': ['pv'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12', 'tag': None, 'wait_for_package': False} 2024-08-25T19:43:28.449 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=db4959e44f0252a53b4d8688fbf88b3dda70ea12 2024-08-25T19:43:28.590 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/db4959e44f0252a53b4d8688fbf88b3dda70ea12/centos/9/flavors/default/ 2024-08-25T19:43:28.591 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5971.gdb4959e4 2024-08-25T19:43:28.594 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/db4959e44f0252a53b4d8688fbf88b3dda70ea12/centos/9/flavors/default/ 2024-08-25T19:43:28.594 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5971.gdb4959e4 2024-08-25T19:43:28.702 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/db4959e44f0252a53b4d8688fbf88b3dda70ea12/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/db4959e44f0252a53b4d8688fbf88b3dda70ea12/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/db4959e44f0252a53b4d8688fbf88b3dda70ea12/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-25T19:43:28.702 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:43:28.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-25T19:43:28.711 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/db4959e44f0252a53b4d8688fbf88b3dda70ea12/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/db4959e44f0252a53b4d8688fbf88b3dda70ea12/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/db4959e44f0252a53b4d8688fbf88b3dda70ea12/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-25T19:43:28.711 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:43:28.711 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-25T19:43:28.759 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-mirror, pv, pv, pv on remote rpm x86_64 2024-08-25T19:43:28.759 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/db4959e44f0252a53b4d8688fbf88b3dda70ea12/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-25T19:43:28.767 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-mirror, pv, pv, pv, pv on remote rpm x86_64 2024-08-25T19:43:28.768 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/db4959e44f0252a53b4d8688fbf88b3dda70ea12/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-25T19:43:28.856 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-25T19:43:28.864 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-25T19:43:28.933 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-25T19:43:28.943 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-25T19:43:28.987 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2024-08-25T19:43:28.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2024-08-25T19:43:28.998 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2024-08-25T19:43:29.001 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2024-08-25T19:43:29.264 INFO:teuthology.orchestra.run.smithi119.stdout:45 files removed 2024-08-25T19:43:29.273 INFO:teuthology.orchestra.run.smithi121.stdout:45 files removed 2024-08-25T19:43:29.297 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd rbd-mirror pv pv pv pv 2024-08-25T19:43:29.306 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd rbd-mirror pv pv pv pv 2024-08-25T19:43:29.967 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 299 kB/s | 84 kB 00:00 2024-08-25T19:43:29.990 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 285 kB/s | 84 kB 00:00 2024-08-25T19:43:30.268 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-08-25T19:43:30.300 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 91 kB/s | 24 kB 00:00 2024-08-25T19:43:30.552 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-08-25T19:43:30.583 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-08-25T19:43:32.392 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-08-25T19:43:34.735 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.2 MB 00:04 2024-08-25T19:43:35.843 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 8.8 MB/s | 20 MB 00:02 2024-08-25T19:43:37.861 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-08-25T19:43:40.585 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.6 MB 00:00 2024-08-25T19:43:42.194 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 18 kB 00:00 2024-08-25T19:43:43.149 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 27 MB/s | 22 MB 00:00 2024-08-25T19:43:43.753 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.6 MB 00:01 2024-08-25T19:43:46.079 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 18 kB 00:00 2024-08-25T19:43:47.108 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 25 MB/s | 22 MB 00:00 2024-08-25T19:43:49.438 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-08-25T19:43:51.817 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-25T19:43:51.817 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-25T19:43:51.881 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-08-25T19:43:51.901 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-08-25T19:43:51.901 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-08-25T19:43:51.901 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-08-25T19:43:51.901 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-08-25T19:43:51.901 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 6.6 k 2024-08-25T19:43:51.901 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 5.5 M 2024-08-25T19:43:51.901 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 900 k 2024-08-25T19:43:51.901 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 145 k 2024-08-25T19:43:51.902 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 1.5 M 2024-08-25T19:43:51.902 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 154 k 2024-08-25T19:43:51.902 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 14 M 2024-08-25T19:43:51.902 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 7.4 M 2024-08-25T19:43:51.902 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 49 k 2024-08-25T19:43:51.902 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 11 M 2024-08-25T19:43:51.902 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 50 M 2024-08-25T19:43:51.902 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 285 k 2024-08-25T19:43:51.902 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 781 k 2024-08-25T19:43:51.902 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 32 k 2024-08-25T19:43:51.902 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 755 k 2024-08-25T19:43:51.902 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 127 k 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout: pv x86_64 1.6.20-1.el9 epel 65 k 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 162 k 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 321 k 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 306 k 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 99 k 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 85 k 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 3.2 M 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 170 k 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 3.5 M 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 3.2 M 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-25T19:43:51.903 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-25T19:43:51.904 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 22 M 2024-08-25T19:43:51.904 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 34 k 2024-08-25T19:43:51.904 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 2.4 M 2024-08-25T19:43:51.904 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 287 k 2024-08-25T19:43:51.904 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 4.8 M 2024-08-25T19:43:51.904 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 17 M 2024-08-25T19:43:51.904 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 16 k 2024-08-25T19:43:51.904 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 25 k 2024-08-25T19:43:51.904 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-25T19:43:51.904 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-25T19:43:51.904 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-25T19:43:51.904 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-25T19:43:51.905 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-25T19:43:51.905 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-25T19:43:51.905 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-25T19:43:51.905 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-25T19:43:51.905 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 161 k 2024-08-25T19:43:51.905 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-25T19:43:51.905 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-25T19:43:51.905 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-25T19:43:51.905 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-25T19:43:51.905 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-25T19:43:51.905 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 508 k 2024-08-25T19:43:51.905 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 5.5 M 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.26-1.el9 appstream 38 k 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp x86_64 0.3.26-1.el9 appstream 4.9 M 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-25T19:43:51.906 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 45 k 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 141 k 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel x86_64 3.9.19-7.el9 appstream 245 k 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-25T19:43:51.907 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-25T19:43:51.908 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-25T19:43:51.909 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-25T19:43:51.910 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-25T19:43:51.911 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-25T19:43:51.911 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-25T19:43:51.911 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-25T19:43:51.911 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-25T19:43:51.911 INFO:teuthology.orchestra.run.smithi121.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-08-25T19:43:51.911 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-25T19:43:51.911 INFO:teuthology.orchestra.run.smithi121.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-25T19:43:51.911 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-25T19:43:51.911 INFO:teuthology.orchestra.run.smithi121.stdout:Installing weak dependencies: 2024-08-25T19:43:51.911 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-25T19:43:51.912 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:43:51.912 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-08-25T19:43:51.912 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-08-25T19:43:51.912 INFO:teuthology.orchestra.run.smithi121.stdout:Install 125 Packages 2024-08-25T19:43:51.912 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 2 Packages 2024-08-25T19:43:51.912 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:43:51.912 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 215 M 2024-08-25T19:43:51.912 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-08-25T19:43:53.022 INFO:teuthology.orchestra.run.smithi121.stdout:(1/127): ceph-19.0.0-5971.gdb4959e4.el9.x86_64. 45 kB/s | 6.6 kB 00:00 2024-08-25T19:43:53.181 INFO:teuthology.orchestra.run.smithi121.stdout:(2/127): ceph-fuse-19.0.0-5971.gdb4959e4.el9.x8 5.6 MB/s | 900 kB 00:00 2024-08-25T19:43:53.264 INFO:teuthology.orchestra.run.smithi121.stdout:(3/127): ceph-immutable-object-cache-19.0.0-597 1.7 MB/s | 145 kB 00:00 2024-08-25T19:43:53.390 INFO:teuthology.orchestra.run.smithi121.stdout:(4/127): ceph-base-19.0.0-5971.gdb4959e4.el9.x8 11 MB/s | 5.5 MB 00:00 2024-08-25T19:43:53.430 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-08-25T19:43:53.490 INFO:teuthology.orchestra.run.smithi121.stdout:(5/127): ceph-mgr-19.0.0-5971.gdb4959e4.el9.x86 15 MB/s | 1.5 MB 00:00 2024-08-25T19:43:53.532 INFO:teuthology.orchestra.run.smithi121.stdout:(6/127): ceph-mds-19.0.0-5971.gdb4959e4.el9.x86 8.9 MB/s | 2.4 MB 00:00 2024-08-25T19:43:53.791 INFO:teuthology.orchestra.run.smithi121.stdout:(7/127): ceph-mon-19.0.0-5971.gdb4959e4.el9.x86 16 MB/s | 4.8 MB 00:00 2024-08-25T19:43:54.124 INFO:teuthology.orchestra.run.smithi121.stdout:(8/127): ceph-common-19.0.0-5971.gdb4959e4.el9. 17 MB/s | 22 MB 00:01 2024-08-25T19:43:54.191 INFO:teuthology.orchestra.run.smithi121.stdout:(9/127): ceph-selinux-19.0.0-5971.gdb4959e4.el9 380 kB/s | 25 kB 00:00 2024-08-25T19:43:54.724 INFO:teuthology.orchestra.run.smithi121.stdout:(10/127): ceph-osd-19.0.0-5971.gdb4959e4.el9.x8 14 MB/s | 17 MB 00:01 2024-08-25T19:43:54.783 INFO:teuthology.orchestra.run.smithi121.stdout:(11/127): libcephfs-devel-19.0.0-5971.gdb4959e4 546 kB/s | 32 kB 00:00 2024-08-25T19:43:54.892 INFO:teuthology.orchestra.run.smithi121.stdout:(12/127): libcephfs2-19.0.0-5971.gdb4959e4.el9. 6.8 MB/s | 755 kB 00:00 2024-08-25T19:43:54.967 INFO:teuthology.orchestra.run.smithi121.stdout:(13/127): libcephsqlite-19.0.0-5971.gdb4959e4.e 2.1 MB/s | 161 kB 00:00 2024-08-25T19:43:55.054 INFO:teuthology.orchestra.run.smithi121.stdout:(14/127): librados-devel-19.0.0-5971.gdb4959e4. 1.4 MB/s | 127 kB 00:00 2024-08-25T19:43:55.218 INFO:teuthology.orchestra.run.smithi121.stdout:(15/127): ceph-radosgw-19.0.0-5971.gdb4959e4.el 7.7 MB/s | 11 MB 00:01 2024-08-25T19:43:55.251 INFO:teuthology.orchestra.run.smithi121.stdout:(16/127): libradosstriper1-19.0.0-5971.gdb4959e 2.5 MB/s | 508 kB 00:00 2024-08-25T19:43:55.318 INFO:teuthology.orchestra.run.smithi121.stdout:(17/127): python3-ceph-argparse-19.0.0-5971.gdb 679 kB/s | 45 kB 00:00 2024-08-25T19:43:55.377 INFO:teuthology.orchestra.run.smithi121.stdout:(18/127): python3-ceph-common-19.0.0-5971.gdb49 2.4 MB/s | 141 kB 00:00 2024-08-25T19:43:55.444 INFO:teuthology.orchestra.run.smithi121.stdout:(19/127): python3-cephfs-19.0.0-5971.gdb4959e4. 2.4 MB/s | 162 kB 00:00 2024-08-25T19:43:55.519 INFO:teuthology.orchestra.run.smithi121.stdout:(20/127): python3-rados-19.0.0-5971.gdb4959e4.e 4.2 MB/s | 321 kB 00:00 2024-08-25T19:43:55.594 INFO:teuthology.orchestra.run.smithi121.stdout:(21/127): python3-rbd-19.0.0-5971.gdb4959e4.el9 4.0 MB/s | 306 kB 00:00 2024-08-25T19:43:55.661 INFO:teuthology.orchestra.run.smithi121.stdout:(22/127): python3-rgw-19.0.0-5971.gdb4959e4.el9 1.5 MB/s | 99 kB 00:00 2024-08-25T19:43:55.745 INFO:teuthology.orchestra.run.smithi121.stdout:(23/127): librgw2-19.0.0-5971.gdb4959e4.el9.x86 10 MB/s | 5.5 MB 00:00 2024-08-25T19:43:55.751 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-25T19:43:55.752 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-25T19:43:55.762 INFO:teuthology.orchestra.run.smithi121.stdout:(24/127): rbd-fuse-19.0.0-5971.gdb4959e4.el9.x8 850 kB/s | 85 kB 00:00 2024-08-25T19:43:55.804 INFO:teuthology.orchestra.run.smithi121.stdout:(25/127): rbd-nbd-19.0.0-5971.gdb4959e4.el9.x86 4.0 MB/s | 170 kB 00:00 2024-08-25T19:43:55.816 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-08-25T19:43:55.829 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-08-25T19:43:55.829 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-08-25T19:43:55.829 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-08-25T19:43:55.829 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 6.6 k 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 5.5 M 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 900 k 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 145 k 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 1.5 M 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 154 k 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 14 M 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 7.4 M 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 49 k 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 11 M 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 50 M 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 285 k 2024-08-25T19:43:55.830 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 781 k 2024-08-25T19:43:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 32 k 2024-08-25T19:43:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 755 k 2024-08-25T19:43:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 127 k 2024-08-25T19:43:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: pv x86_64 1.6.20-1.el9 epel 65 k 2024-08-25T19:43:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 162 k 2024-08-25T19:43:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 321 k 2024-08-25T19:43:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 306 k 2024-08-25T19:43:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 99 k 2024-08-25T19:43:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 85 k 2024-08-25T19:43:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 3.2 M 2024-08-25T19:43:55.831 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 170 k 2024-08-25T19:43:55.831 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 3.5 M 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 3.2 M 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 22 M 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 34 k 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 2.4 M 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 287 k 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 4.8 M 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 17 M 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5971.gdb4959e4.el9 ceph-noarch 16 k 2024-08-25T19:43:55.832 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 25 k 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 161 k 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-25T19:43:55.833 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-25T19:43:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-25T19:43:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 508 k 2024-08-25T19:43:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-25T19:43:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 5.5 M 2024-08-25T19:43:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-25T19:43:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-25T19:43:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-25T19:43:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-25T19:43:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-25T19:43:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.26-1.el9 appstream 38 k 2024-08-25T19:43:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.26-1.el9 appstream 4.9 M 2024-08-25T19:43:55.834 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 45 k 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:19.0.0-5971.gdb4959e4.el9 ceph 141 k 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-25T19:43:55.835 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.19-7.el9 appstream 245 k 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-25T19:43:55.836 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-25T19:43:55.837 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-25T19:43:55.838 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2024-08-25T19:43:55.839 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-25T19:43:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:43:55.840 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-08-25T19:43:55.840 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-08-25T19:43:55.840 INFO:teuthology.orchestra.run.smithi119.stdout:Install 125 Packages 2024-08-25T19:43:55.840 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 2 Packages 2024-08-25T19:43:55.840 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:43:55.840 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 215 M 2024-08-25T19:43:55.840 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-08-25T19:43:56.288 INFO:teuthology.orchestra.run.smithi121.stdout:(26/127): ceph-test-19.0.0-5971.gdb4959e4.el9.x 24 MB/s | 50 MB 00:02 2024-08-25T19:43:56.330 INFO:teuthology.orchestra.run.smithi121.stdout:(27/127): ceph-grafana-dashboards-19.0.0-5971.g 65 kB/s | 34 kB 00:00 2024-08-25T19:43:56.355 INFO:teuthology.orchestra.run.smithi121.stdout:(28/127): ceph-mgr-cephadm-19.0.0-5971.gdb4959e 2.3 MB/s | 154 kB 00:00 2024-08-25T19:43:56.430 INFO:teuthology.orchestra.run.smithi121.stdout:(29/127): rbd-mirror-19.0.0-5971.gdb4959e4.el9. 4.6 MB/s | 3.2 MB 00:00 2024-08-25T19:43:56.465 INFO:teuthology.orchestra.run.smithi119.stdout:(1/127): ceph-19.0.0-5971.gdb4959e4.el9.x86_64. 50 kB/s | 6.6 kB 00:00 2024-08-25T19:43:56.480 INFO:teuthology.orchestra.run.smithi121.stdout:(30/127): ceph-mgr-modules-core-19.0.0-5971.gdb 5.7 MB/s | 287 kB 00:00 2024-08-25T19:43:56.564 INFO:teuthology.orchestra.run.smithi121.stdout:(31/127): ceph-mgr-rook-19.0.0-5971.gdb4959e4.e 594 kB/s | 49 kB 00:00 2024-08-25T19:43:56.656 INFO:teuthology.orchestra.run.smithi121.stdout:(32/127): ceph-prometheus-alerts-19.0.0-5971.gd 179 kB/s | 16 kB 00:00 2024-08-25T19:43:56.765 INFO:teuthology.orchestra.run.smithi121.stdout:(33/127): ceph-mgr-diskprediction-local-19.0.0- 18 MB/s | 7.4 MB 00:00 2024-08-25T19:43:56.790 INFO:teuthology.orchestra.run.smithi121.stdout:(34/127): ceph-volume-19.0.0-5971.gdb4959e4.el9 2.1 MB/s | 285 kB 00:00 2024-08-25T19:43:56.840 INFO:teuthology.orchestra.run.smithi121.stdout:(35/127): cephadm-19.0.0-5971.gdb4959e4.el9.noa 10 MB/s | 781 kB 00:00 2024-08-25T19:43:56.857 INFO:teuthology.orchestra.run.smithi119.stdout:(2/127): ceph-fuse-19.0.0-5971.gdb4959e4.el9.x8 2.2 MB/s | 900 kB 00:00 2024-08-25T19:43:56.873 INFO:teuthology.orchestra.run.smithi121.stdout:(36/127): fuse-2.9.9-16.el9.x86_64.rpm 957 kB/s | 80 kB 00:00 2024-08-25T19:43:56.932 INFO:teuthology.orchestra.run.smithi121.stdout:(37/127): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-08-25T19:43:56.941 INFO:teuthology.orchestra.run.smithi119.stdout:(3/127): ceph-immutable-object-cache-19.0.0-597 1.7 MB/s | 145 kB 00:00 2024-08-25T19:43:56.991 INFO:teuthology.orchestra.run.smithi121.stdout:(38/127): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-08-25T19:43:57.024 INFO:teuthology.orchestra.run.smithi121.stdout:(39/127): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-08-25T19:43:57.191 INFO:teuthology.orchestra.run.smithi121.stdout:(40/127): ceph-mgr-dashboard-19.0.0-5971.gdb495 16 MB/s | 14 MB 00:00 2024-08-25T19:43:57.217 INFO:teuthology.orchestra.run.smithi121.stdout:(41/127): python3-ply-3.11-14.el9.noarch.rpm 555 kB/s | 106 kB 00:00 2024-08-25T19:43:57.258 INFO:teuthology.orchestra.run.smithi121.stdout:(42/127): python3-cryptography-36.0.1-4.el9.x86 4.6 MB/s | 1.2 MB 00:00 2024-08-25T19:43:57.284 INFO:teuthology.orchestra.run.smithi121.stdout:(43/127): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-08-25T19:43:57.309 INFO:teuthology.orchestra.run.smithi121.stdout:(44/127): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-08-25T19:43:57.324 INFO:teuthology.orchestra.run.smithi119.stdout:(4/127): ceph-mds-19.0.0-5971.gdb4959e4.el9.x86 6.2 MB/s | 2.4 MB 00:00 2024-08-25T19:43:57.335 INFO:teuthology.orchestra.run.smithi121.stdout:(45/127): python3-pycparser-2.20-6.el9.noarch.r 947 kB/s | 135 kB 00:00 2024-08-25T19:43:57.377 INFO:teuthology.orchestra.run.smithi121.stdout:(46/127): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-08-25T19:43:57.399 INFO:teuthology.orchestra.run.smithi119.stdout:(5/127): ceph-base-19.0.0-5971.gdb4959e4.el9.x8 5.2 MB/s | 5.5 MB 00:01 2024-08-25T19:43:57.402 INFO:teuthology.orchestra.run.smithi121.stdout:(47/127): boost-program-options-1.75.0-8.el9.x8 908 kB/s | 107 kB 00:00 2024-08-25T19:43:57.443 INFO:teuthology.orchestra.run.smithi121.stdout:(48/127): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-08-25T19:43:57.484 INFO:teuthology.orchestra.run.smithi121.stdout:(49/127): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-08-25T19:43:57.527 INFO:teuthology.orchestra.run.smithi121.stdout:(50/127): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-08-25T19:43:57.533 INFO:teuthology.orchestra.run.smithi119.stdout:(6/127): ceph-mgr-19.0.0-5971.gdb4959e4.el9.x86 7.3 MB/s | 1.5 MB 00:00 2024-08-25T19:43:57.569 INFO:teuthology.orchestra.run.smithi121.stdout:(51/127): librdkafka-1.6.1-102.el9.x86_64.rpm 7.6 MB/s | 662 kB 00:00 2024-08-25T19:43:57.603 INFO:teuthology.orchestra.run.smithi121.stdout:(52/127): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-08-25T19:43:57.629 INFO:teuthology.orchestra.run.smithi121.stdout:(53/127): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-08-25T19:43:57.671 INFO:teuthology.orchestra.run.smithi121.stdout:(54/127): openblas-0.3.26-1.el9.x86_64.rpm 575 kB/s | 38 kB 00:00 2024-08-25T19:43:57.738 INFO:teuthology.orchestra.run.smithi121.stdout:(55/127): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-08-25T19:43:57.742 INFO:teuthology.orchestra.run.smithi119.stdout:(7/127): ceph-mon-19.0.0-5971.gdb4959e4.el9.x86 14 MB/s | 4.8 MB 00:00 2024-08-25T19:43:57.771 INFO:teuthology.orchestra.run.smithi121.stdout:(56/127): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-08-25T19:43:57.863 INFO:teuthology.orchestra.run.smithi121.stdout:(57/127): openblas-openmp-0.3.26-1.el9.x86_64.r 21 MB/s | 4.9 MB 00:00 2024-08-25T19:43:57.913 INFO:teuthology.orchestra.run.smithi121.stdout:(58/127): python3-devel-3.9.19-7.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-08-25T19:43:57.939 INFO:teuthology.orchestra.run.smithi121.stdout:(59/127): python3-jinja2-2.11.3-6.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-08-25T19:43:57.964 INFO:teuthology.orchestra.run.smithi121.stdout:(60/127): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-08-25T19:43:57.989 INFO:teuthology.orchestra.run.smithi121.stdout:(61/127): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-08-25T19:43:58.015 INFO:teuthology.orchestra.run.smithi121.stdout:(62/127): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-08-25T19:43:58.040 INFO:teuthology.orchestra.run.smithi121.stdout:(63/127): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-08-25T19:43:58.149 INFO:teuthology.orchestra.run.smithi121.stdout:(64/127): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-08-25T19:43:58.173 INFO:teuthology.orchestra.run.smithi121.stdout:(65/127): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-08-25T19:43:58.199 INFO:teuthology.orchestra.run.smithi121.stdout:(66/127): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-08-25T19:43:58.224 INFO:teuthology.orchestra.run.smithi121.stdout:(67/127): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-25T19:43:58.250 INFO:teuthology.orchestra.run.smithi121.stdout:(68/127): python3-prettytable-0.7.2-27.el9.noar 837 kB/s | 42 kB 00:00 2024-08-25T19:43:58.275 INFO:teuthology.orchestra.run.smithi121.stdout:(69/127): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-08-25T19:43:58.300 INFO:teuthology.orchestra.run.smithi121.stdout:(70/127): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-08-25T19:43:58.325 INFO:teuthology.orchestra.run.smithi121.stdout:(71/127): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-08-25T19:43:58.350 INFO:teuthology.orchestra.run.smithi121.stdout:(72/127): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-25T19:43:58.375 INFO:teuthology.orchestra.run.smithi121.stdout:(73/127): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-25T19:43:58.376 INFO:teuthology.orchestra.run.smithi119.stdout:(8/127): ceph-common-19.0.0-5971.gdb4959e4.el9. 11 MB/s | 22 MB 00:02 2024-08-25T19:43:58.426 INFO:teuthology.orchestra.run.smithi119.stdout:(9/127): ceph-selinux-19.0.0-5971.gdb4959e4.el9 507 kB/s | 25 kB 00:00 2024-08-25T19:43:58.460 INFO:teuthology.orchestra.run.smithi121.stdout:(74/127): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-08-25T19:43:58.485 INFO:teuthology.orchestra.run.smithi121.stdout:(75/127): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-08-25T19:43:58.510 INFO:teuthology.orchestra.run.smithi121.stdout:(76/127): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-08-25T19:43:58.527 INFO:teuthology.orchestra.run.smithi121.stdout:(77/127): qatlib-service-24.02.0-1.el9.x86_64.r 853 kB/s | 36 kB 00:00 2024-08-25T19:43:58.544 INFO:teuthology.orchestra.run.smithi121.stdout:(78/127): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-08-25T19:43:58.569 INFO:teuthology.orchestra.run.smithi121.stdout:(79/127): socat-1.7.4.1-6.el9.x86_64.rpm 7.1 MB/s | 304 kB 00:00 2024-08-25T19:43:58.576 INFO:teuthology.orchestra.run.smithi119.stdout:(10/127): ceph-radosgw-19.0.0-5971.gdb4959e4.el 13 MB/s | 11 MB 00:00 2024-08-25T19:43:58.586 INFO:teuthology.orchestra.run.smithi121.stdout:(80/127): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-08-25T19:43:58.618 INFO:teuthology.orchestra.run.smithi119.stdout:(11/127): libcephfs-devel-19.0.0-5971.gdb4959e4 765 kB/s | 32 kB 00:00 2024-08-25T19:43:58.868 INFO:teuthology.orchestra.run.smithi119.stdout:(12/127): ceph-osd-19.0.0-5971.gdb4959e4.el9.x8 13 MB/s | 17 MB 00:01 2024-08-25T19:43:58.870 INFO:teuthology.orchestra.run.smithi121.stdout:(81/127): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-08-25T19:43:58.902 INFO:teuthology.orchestra.run.smithi119.stdout:(13/127): libcephfs2-19.0.0-5971.gdb4959e4.el9. 2.6 MB/s | 755 kB 00:00 2024-08-25T19:43:58.920 INFO:teuthology.orchestra.run.smithi121.stdout:(82/127): abseil-cpp-20211102.0-3.el9.x86_64.rp 11 MB/s | 548 kB 00:00 2024-08-25T19:43:58.927 INFO:teuthology.orchestra.run.smithi119.stdout:(14/127): libcephsqlite-19.0.0-5971.gdb4959e4.e 2.7 MB/s | 161 kB 00:00 2024-08-25T19:43:58.952 INFO:teuthology.orchestra.run.smithi119.stdout:(15/127): librados-devel-19.0.0-5971.gdb4959e4. 2.5 MB/s | 127 kB 00:00 2024-08-25T19:43:58.954 INFO:teuthology.orchestra.run.smithi121.stdout:(83/127): gperftools-libs-2.9.1-2.el9.x86_64.rp 9.0 MB/s | 309 kB 00:00 2024-08-25T19:43:58.986 INFO:teuthology.orchestra.run.smithi119.stdout:(16/127): libradosstriper1-19.0.0-5971.gdb4959e 8.5 MB/s | 508 kB 00:00 2024-08-25T19:43:58.996 INFO:teuthology.orchestra.run.smithi121.stdout:(84/127): grpc-data-1.46.7-10.el9.noarch.rpm 856 kB/s | 19 kB 00:00 2024-08-25T19:43:59.036 INFO:teuthology.orchestra.run.smithi119.stdout:(17/127): python3-ceph-argparse-19.0.0-5971.gdb 907 kB/s | 45 kB 00:00 2024-08-25T19:43:59.096 INFO:teuthology.orchestra.run.smithi121.stdout:(85/127): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-08-25T19:43:59.103 INFO:teuthology.orchestra.run.smithi119.stdout:(18/127): python3-ceph-common-19.0.0-5971.gdb49 2.1 MB/s | 141 kB 00:00 2024-08-25T19:43:59.122 INFO:teuthology.orchestra.run.smithi121.stdout:(86/127): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-25T19:43:59.147 INFO:teuthology.orchestra.run.smithi121.stdout:(87/127): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-25T19:43:59.172 INFO:teuthology.orchestra.run.smithi121.stdout:(88/127): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-25T19:43:59.178 INFO:teuthology.orchestra.run.smithi119.stdout:(19/127): python3-cephfs-19.0.0-5971.gdb4959e4. 2.1 MB/s | 162 kB 00:00 2024-08-25T19:43:59.197 INFO:teuthology.orchestra.run.smithi121.stdout:(89/127): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-08-25T19:43:59.239 INFO:teuthology.orchestra.run.smithi121.stdout:(90/127): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-08-25T19:43:59.262 INFO:teuthology.orchestra.run.smithi119.stdout:(20/127): librgw2-19.0.0-5971.gdb4959e4.el9.x86 18 MB/s | 5.5 MB 00:00 2024-08-25T19:43:59.265 INFO:teuthology.orchestra.run.smithi121.stdout:(91/127): pv-1.6.20-1.el9.x86_64.rpm 2.5 MB/s | 65 kB 00:00 2024-08-25T19:43:59.287 INFO:teuthology.orchestra.run.smithi119.stdout:(21/127): python3-rados-19.0.0-5971.gdb4959e4.e 2.9 MB/s | 321 kB 00:00 2024-08-25T19:43:59.290 INFO:teuthology.orchestra.run.smithi121.stdout:(92/127): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-08-25T19:43:59.313 INFO:teuthology.orchestra.run.smithi119.stdout:(22/127): python3-rbd-19.0.0-5971.gdb4959e4.el9 5.9 MB/s | 306 kB 00:00 2024-08-25T19:43:59.332 INFO:teuthology.orchestra.run.smithi121.stdout:(93/127): python3-asyncssh-2.13.2-5.el9.noarch. 8.1 MB/s | 548 kB 00:00 2024-08-25T19:43:59.338 INFO:teuthology.orchestra.run.smithi119.stdout:(23/127): python3-rgw-19.0.0-5971.gdb4959e4.el9 1.9 MB/s | 99 kB 00:00 2024-08-25T19:43:59.357 INFO:teuthology.orchestra.run.smithi121.stdout:(94/127): python3-cachetools-4.2.4-1.el9.noarch 1.4 MB/s | 32 kB 00:00 2024-08-25T19:43:59.363 INFO:teuthology.orchestra.run.smithi119.stdout:(24/127): rbd-fuse-19.0.0-5971.gdb4959e4.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-25T19:43:59.382 INFO:teuthology.orchestra.run.smithi121.stdout:(95/127): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 472 kB/s | 43 kB 00:00 2024-08-25T19:43:59.408 INFO:teuthology.orchestra.run.smithi121.stdout:(96/127): python3-certifi-2023.05.07-4.el9.noar 281 kB/s | 14 kB 00:00 2024-08-25T19:43:59.422 INFO:teuthology.orchestra.run.smithi119.stdout:(25/127): rbd-nbd-19.0.0-5971.gdb4959e4.el9.x86 2.9 MB/s | 170 kB 00:00 2024-08-25T19:43:59.433 INFO:teuthology.orchestra.run.smithi121.stdout:(97/127): python3-cheroot-8.6.0-4.el9.noarch.rp 3.3 MB/s | 172 kB 00:00 2024-08-25T19:43:59.475 INFO:teuthology.orchestra.run.smithi121.stdout:(98/127): python3-cherrypy-18.6.1-2.el9.noarch. 5.2 MB/s | 358 kB 00:00 2024-08-25T19:43:59.497 INFO:teuthology.orchestra.run.smithi119.stdout:(26/127): ceph-grafana-dashboards-19.0.0-5971.g 460 kB/s | 34 kB 00:00 2024-08-25T19:43:59.500 INFO:teuthology.orchestra.run.smithi121.stdout:(99/127): python3-google-auth-2.34.0-1.el9.noar 3.2 MB/s | 220 kB 00:00 2024-08-25T19:43:59.534 INFO:teuthology.orchestra.run.smithi121.stdout:(100/127): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-08-25T19:43:59.556 INFO:teuthology.orchestra.run.smithi119.stdout:(27/127): rbd-mirror-19.0.0-5971.gdb4959e4.el9. 15 MB/s | 3.2 MB 00:00 2024-08-25T19:43:59.559 INFO:teuthology.orchestra.run.smithi121.stdout:(101/127): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-08-25T19:43:59.581 INFO:teuthology.orchestra.run.smithi119.stdout:(28/127): ceph-mgr-cephadm-19.0.0-5971.gdb4959e 1.8 MB/s | 154 kB 00:00 2024-08-25T19:43:59.585 INFO:teuthology.orchestra.run.smithi121.stdout:(102/127): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-08-25T19:43:59.610 INFO:teuthology.orchestra.run.smithi121.stdout:(103/127): python3-jaraco-collections-3.0.0-8.e 940 kB/s | 23 kB 00:00 2024-08-25T19:43:59.635 INFO:teuthology.orchestra.run.smithi121.stdout:(104/127): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-08-25T19:43:59.661 INFO:teuthology.orchestra.run.smithi121.stdout:(105/127): python3-jaraco-text-3.2.0-6.el9.noar 784 kB/s | 20 kB 00:00 2024-08-25T19:43:59.702 INFO:teuthology.orchestra.run.smithi121.stdout:(106/127): protobuf-compiler-3.14.0-13.el9.x86_ 774 kB/s | 863 kB 00:01 2024-08-25T19:43:59.745 INFO:teuthology.orchestra.run.smithi121.stdout:(107/127): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-08-25T19:43:59.778 INFO:teuthology.orchestra.run.smithi121.stdout:(108/127): python3-more-itertools-8.12.0-2.el9. 3.2 MB/s | 79 kB 00:00 2024-08-25T19:43:59.812 INFO:teuthology.orchestra.run.smithi121.stdout:(109/127): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-08-25T19:43:59.837 INFO:teuthology.orchestra.run.smithi121.stdout:(110/127): python3-logutils-0.3.5-21.el9.noarch 344 kB/s | 46 kB 00:00 2024-08-25T19:43:59.863 INFO:teuthology.orchestra.run.smithi121.stdout:(111/127): python3-natsort-7.1.1-5.el9.noarch.r 703 kB/s | 58 kB 00:00 2024-08-25T19:43:59.888 INFO:teuthology.orchestra.run.smithi121.stdout:(112/127): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-08-25T19:43:59.930 INFO:teuthology.orchestra.run.smithi121.stdout:(113/127): python3-repoze-lru-0.7-16.el9.noarch 743 kB/s | 31 kB 00:00 2024-08-25T19:43:59.963 INFO:teuthology.orchestra.run.smithi121.stdout:(114/127): python3-pyOpenSSL-21.0.0-1.el9.noarc 896 kB/s | 90 kB 00:00 2024-08-25T19:43:59.981 INFO:teuthology.orchestra.run.smithi121.stdout:(115/127): python3-rsa-4.9-2.el9.noarch.rpm 3.4 MB/s | 59 kB 00:00 2024-08-25T19:44:00.006 INFO:teuthology.orchestra.run.smithi121.stdout:(116/127): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-08-25T19:44:00.022 INFO:teuthology.orchestra.run.smithi121.stdout:(117/127): python3-tempora-5.0.0-2.el9.noarch.r 858 kB/s | 36 kB 00:00 2024-08-25T19:44:00.048 INFO:teuthology.orchestra.run.smithi121.stdout:(118/127): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-08-25T19:44:00.073 INFO:teuthology.orchestra.run.smithi121.stdout:(119/127): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-08-25T19:44:00.098 INFO:teuthology.orchestra.run.smithi121.stdout:(120/127): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-08-25T19:44:00.124 INFO:teuthology.orchestra.run.smithi121.stdout:(121/127): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-08-25T19:44:00.140 INFO:teuthology.orchestra.run.smithi119.stdout:(29/127): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-08-25T19:44:00.140 INFO:teuthology.orchestra.run.smithi121.stdout:(122/127): python3-zc-lockfile-2.0-10.el9.noarc 478 kB/s | 20 kB 00:00 2024-08-25T19:44:00.166 INFO:teuthology.orchestra.run.smithi121.stdout:(123/127): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-08-25T19:44:00.191 INFO:teuthology.orchestra.run.smithi121.stdout:(124/127): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-08-25T19:44:00.207 INFO:teuthology.orchestra.run.smithi119.stdout:(30/127): ceph-mgr-modules-core-19.0.0-5971.gdb 4.2 MB/s | 287 kB 00:00 2024-08-25T19:44:00.241 INFO:teuthology.orchestra.run.smithi121.stdout:(125/127): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-25T19:44:00.282 INFO:teuthology.orchestra.run.smithi119.stdout:(31/127): ceph-mgr-rook-19.0.0-5971.gdb4959e4.e 661 kB/s | 49 kB 00:00 2024-08-25T19:44:00.357 INFO:teuthology.orchestra.run.smithi119.stdout:(32/127): ceph-prometheus-alerts-19.0.0-5971.gd 220 kB/s | 16 kB 00:00 2024-08-25T19:44:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:(33/127): ceph-mgr-dashboard-19.0.0-5971.gdb495 15 MB/s | 14 MB 00:00 2024-08-25T19:44:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:(34/127): ceph-volume-19.0.0-5971.gdb4959e4.el9 1.7 MB/s | 285 kB 00:00 2024-08-25T19:44:00.542 INFO:teuthology.orchestra.run.smithi121.stdout:(126/127): librados2-19.0.0-5971.gdb4959e4.el9. 9.2 MB/s | 3.5 MB 00:00 2024-08-25T19:44:00.592 INFO:teuthology.orchestra.run.smithi121.stdout:(127/127): librbd1-19.0.0-5971.gdb4959e4.el9.x8 8.1 MB/s | 3.2 MB 00:00 2024-08-25T19:44:00.597 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-08-25T19:44:00.597 INFO:teuthology.orchestra.run.smithi121.stdout:Total 25 MB/s | 215 MB 00:08 2024-08-25T19:44:01.017 INFO:teuthology.orchestra.run.smithi119.stdout:(35/127): ceph-test-19.0.0-5971.gdb4959e4.el9.x 19 MB/s | 50 MB 00:02 2024-08-25T19:44:01.051 INFO:teuthology.orchestra.run.smithi119.stdout:(36/127): cephadm-19.0.0-5971.gdb4959e4.el9.noa 1.4 MB/s | 781 kB 00:00 2024-08-25T19:44:01.134 INFO:teuthology.orchestra.run.smithi119.stdout:(37/127): fuse-2.9.9-16.el9.x86_64.rpm 131 kB/s | 80 kB 00:00 2024-08-25T19:44:01.184 INFO:teuthology.orchestra.run.smithi119.stdout:(38/127): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-08-25T19:44:01.235 INFO:teuthology.orchestra.run.smithi119.stdout:(39/127): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-08-25T19:44:01.285 INFO:teuthology.orchestra.run.smithi119.stdout:(40/127): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-08-25T19:44:01.343 INFO:teuthology.orchestra.run.smithi119.stdout:(41/127): python3-ply-3.11-14.el9.noarch.rpm 993 kB/s | 106 kB 00:00 2024-08-25T19:44:01.385 INFO:teuthology.orchestra.run.smithi119.stdout:(42/127): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-08-25T19:44:01.419 INFO:teuthology.orchestra.run.smithi119.stdout:(43/127): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-08-25T19:44:01.478 INFO:teuthology.orchestra.run.smithi119.stdout:(44/127): python3-requests-2.25.1-8.el9.noarch. 936 kB/s | 125 kB 00:00 2024-08-25T19:44:01.536 INFO:teuthology.orchestra.run.smithi119.stdout:(45/127): python3-urllib3-1.26.5-6.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-08-25T19:44:01.746 INFO:teuthology.orchestra.run.smithi119.stdout:(46/127): flexiblas-3.0.4-8.el9.x86_64.rpm 122 kB/s | 33 kB 00:00 2024-08-25T19:44:01.854 INFO:teuthology.orchestra.run.smithi119.stdout:(47/127): flexiblas-openblas-openmp-3.0.4-8.el9 169 kB/s | 18 kB 00:00 2024-08-25T19:44:01.879 INFO:teuthology.orchestra.run.smithi119.stdout:(48/127): boost-program-options-1.75.0-8.el9.x8 238 kB/s | 107 kB 00:00 2024-08-25T19:44:01.925 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-08-25T19:44:01.988 INFO:teuthology.orchestra.run.smithi119.stdout:(49/127): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-08-25T19:44:02.038 INFO:teuthology.orchestra.run.smithi119.stdout:(50/127): libpmemobj-1.12.1-1.el9.x86_64.rpm 871 kB/s | 160 kB 00:00 2024-08-25T19:44:02.051 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-08-25T19:44:02.052 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-08-25T19:44:02.222 INFO:teuthology.orchestra.run.smithi119.stdout:(51/127): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-08-25T19:44:02.331 INFO:teuthology.orchestra.run.smithi119.stdout:(52/127): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-08-25T19:44:02.376 INFO:teuthology.orchestra.run.smithi119.stdout:(53/127): librdkafka-1.6.1-102.el9.x86_64.rpm 1.7 MB/s | 662 kB 00:00 2024-08-25T19:44:02.431 INFO:teuthology.orchestra.run.smithi119.stdout:(54/127): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.3 MB/s | 3.0 MB 00:00 2024-08-25T19:44:02.448 INFO:teuthology.orchestra.run.smithi119.stdout:(55/127): openblas-0.3.26-1.el9.x86_64.rpm 328 kB/s | 38 kB 00:00 2024-08-25T19:44:02.557 INFO:teuthology.orchestra.run.smithi119.stdout:(56/127): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-08-25T19:44:02.666 INFO:teuthology.orchestra.run.smithi119.stdout:(57/127): python3-devel-3.9.19-7.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-08-25T19:44:02.749 INFO:teuthology.orchestra.run.smithi119.stdout:(58/127): openblas-openmp-0.3.26-1.el9.x86_64.r 13 MB/s | 4.9 MB 00:00 2024-08-25T19:44:02.774 INFO:teuthology.orchestra.run.smithi119.stdout:(59/127): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-08-25T19:44:02.866 INFO:teuthology.orchestra.run.smithi119.stdout:(60/127): python3-jsonpatch-1.21-16.el9.noarch. 220 kB/s | 26 kB 00:00 2024-08-25T19:44:02.892 INFO:teuthology.orchestra.run.smithi119.stdout:(61/127): python3-jsonpointer-2.0-4.el9.noarch. 164 kB/s | 19 kB 00:00 2024-08-25T19:44:02.975 INFO:teuthology.orchestra.run.smithi119.stdout:(62/127): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-08-25T19:44:03.000 INFO:teuthology.orchestra.run.smithi119.stdout:(63/127): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-08-25T19:44:03.026 INFO:teuthology.orchestra.run.smithi119.stdout:(64/127): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-08-25T19:44:03.118 INFO:teuthology.orchestra.run.smithi119.stdout:(65/127): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-08-25T19:44:03.143 INFO:teuthology.orchestra.run.smithi119.stdout:(66/127): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-08-25T19:44:03.235 INFO:teuthology.orchestra.run.smithi119.stdout:(67/127): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-08-25T19:44:03.260 INFO:teuthology.orchestra.run.smithi119.stdout:(68/127): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-08-25T19:44:03.277 INFO:teuthology.orchestra.run.smithi119.stdout:(69/127): python3-packaging-20.9-5.el9.noarch.r 485 kB/s | 77 kB 00:00 2024-08-25T19:44:03.344 INFO:teuthology.orchestra.run.smithi119.stdout:(70/127): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-08-25T19:44:03.369 INFO:teuthology.orchestra.run.smithi119.stdout:(71/127): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-08-25T19:44:03.395 INFO:teuthology.orchestra.run.smithi119.stdout:(72/127): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-08-25T19:44:03.445 INFO:teuthology.orchestra.run.smithi119.stdout:(73/127): python3-pytz-2021.1-5.el9.noarch.rpm 505 kB/s | 51 kB 00:00 2024-08-25T19:44:03.471 INFO:teuthology.orchestra.run.smithi119.stdout:(74/127): python3-requests-oauthlib-1.3.0-12.el 532 kB/s | 54 kB 00:00 2024-08-25T19:44:03.546 INFO:teuthology.orchestra.run.smithi119.stdout:(75/127): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-08-25T19:44:03.579 INFO:teuthology.orchestra.run.smithi119.stdout:(76/127): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-08-25T19:44:03.647 INFO:teuthology.orchestra.run.smithi119.stdout:(77/127): qatlib-service-24.02.0-1.el9.x86_64.r 357 kB/s | 36 kB 00:00 2024-08-25T19:44:03.689 INFO:teuthology.orchestra.run.smithi119.stdout:(78/127): qatzip-libs-1.2.0-1.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-08-25T19:44:03.756 INFO:teuthology.orchestra.run.smithi119.stdout:(79/127): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-08-25T19:44:03.797 INFO:teuthology.orchestra.run.smithi119.stdout:(80/127): xmlstarlet-1.6.1-20.el9.x86_64.rpm 588 kB/s | 64 kB 00:00 2024-08-25T19:44:03.889 INFO:teuthology.orchestra.run.smithi119.stdout:(81/127): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-08-25T19:44:04.081 INFO:teuthology.orchestra.run.smithi119.stdout:(82/127): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-08-25T19:44:04.123 INFO:teuthology.orchestra.run.smithi119.stdout:(83/127): protobuf-compiler-3.14.0-13.el9.x86_6 2.6 MB/s | 863 kB 00:00 2024-08-25T19:44:04.163 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-08-25T19:44:04.164 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-08-25T19:44:04.173 INFO:teuthology.orchestra.run.smithi119.stdout:(84/127): abseil-cpp-20211102.0-3.el9.x86_64.rp 1.9 MB/s | 548 kB 00:00 2024-08-25T19:44:04.199 INFO:teuthology.orchestra.run.smithi119.stdout:(85/127): grpc-data-1.46.7-10.el9.noarch.rpm 265 kB/s | 19 kB 00:00 2024-08-25T19:44:04.224 INFO:teuthology.orchestra.run.smithi119.stdout:(86/127): gperftools-libs-2.9.1-2.el9.x86_64.rp 2.1 MB/s | 309 kB 00:00 2024-08-25T19:44:04.241 INFO:teuthology.orchestra.run.smithi119.stdout:(87/127): libarrow-doc-9.0.0-11.el9.noarch.rpm 620 kB/s | 26 kB 00:00 2024-08-25T19:44:04.258 INFO:teuthology.orchestra.run.smithi119.stdout:(88/127): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-08-25T19:44:04.308 INFO:teuthology.orchestra.run.smithi119.stdout:(89/127): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-08-25T19:44:04.333 INFO:teuthology.orchestra.run.smithi119.stdout:(90/127): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-08-25T19:44:04.359 INFO:teuthology.orchestra.run.smithi119.stdout:(91/127): pv-1.6.20-1.el9.x86_64.rpm 2.6 MB/s | 65 kB 00:00 2024-08-25T19:44:04.401 INFO:teuthology.orchestra.run.smithi119.stdout:(92/127): python3-asyncssh-2.13.2-5.el9.noarch. 13 MB/s | 548 kB 00:00 2024-08-25T19:44:04.434 INFO:teuthology.orchestra.run.smithi119.stdout:(93/127): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-08-25T19:44:04.460 INFO:teuthology.orchestra.run.smithi119.stdout:(94/127): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 741 kB/s | 43 kB 00:00 2024-08-25T19:44:04.485 INFO:teuthology.orchestra.run.smithi119.stdout:(95/127): python3-cachetools-4.2.4-1.el9.noarch 640 kB/s | 32 kB 00:00 2024-08-25T19:44:04.510 INFO:teuthology.orchestra.run.smithi119.stdout:(96/127): python3-certifi-2023.05.07-4.el9.noar 282 kB/s | 14 kB 00:00 2024-08-25T19:44:04.544 INFO:teuthology.orchestra.run.smithi119.stdout:(97/127): python3-cheroot-8.6.0-4.el9.noarch.rp 2.9 MB/s | 172 kB 00:00 2024-08-25T19:44:04.577 INFO:teuthology.orchestra.run.smithi119.stdout:(98/127): python3-cherrypy-18.6.1-2.el9.noarch. 5.2 MB/s | 358 kB 00:00 2024-08-25T19:44:04.603 INFO:teuthology.orchestra.run.smithi119.stdout:(99/127): python3-google-auth-2.34.0-1.el9.noar 3.7 MB/s | 220 kB 00:00 2024-08-25T19:44:04.636 INFO:teuthology.orchestra.run.smithi119.stdout:(100/127): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-08-25T19:44:04.662 INFO:teuthology.orchestra.run.smithi119.stdout:(101/127): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-08-25T19:44:04.696 INFO:teuthology.orchestra.run.smithi119.stdout:(102/127): python3-jaraco-classes-3.2.1-5.el9.n 526 kB/s | 18 kB 00:00 2024-08-25T19:44:04.721 INFO:teuthology.orchestra.run.smithi119.stdout:(103/127): python3-jaraco-collections-3.0.0-8.e 931 kB/s | 23 kB 00:00 2024-08-25T19:44:04.746 INFO:teuthology.orchestra.run.smithi119.stdout:(104/127): python3-jaraco-functools-3.5.0-2.el9 786 kB/s | 19 kB 00:00 2024-08-25T19:44:04.771 INFO:teuthology.orchestra.run.smithi119.stdout:(105/127): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-08-25T19:44:04.830 INFO:teuthology.orchestra.run.smithi119.stdout:(106/127): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-08-25T19:44:04.855 INFO:teuthology.orchestra.run.smithi119.stdout:(107/127): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-25T19:44:04.881 INFO:teuthology.orchestra.run.smithi119.stdout:(108/127): python3-more-itertools-8.12.0-2.el9. 3.0 MB/s | 79 kB 00:00 2024-08-25T19:44:04.906 INFO:teuthology.orchestra.run.smithi119.stdout:(109/127): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-25T19:44:04.956 INFO:teuthology.orchestra.run.smithi119.stdout:(110/127): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-08-25T19:44:04.982 INFO:teuthology.orchestra.run.smithi119.stdout:(111/127): python3-portend-3.1.0-2.el9.noarch.r 665 kB/s | 16 kB 00:00 2024-08-25T19:44:05.007 INFO:teuthology.orchestra.run.smithi119.stdout:(112/127): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-08-25T19:44:05.032 INFO:teuthology.orchestra.run.smithi119.stdout:(113/127): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-08-25T19:44:05.057 INFO:teuthology.orchestra.run.smithi119.stdout:(114/127): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-08-25T19:44:05.083 INFO:teuthology.orchestra.run.smithi119.stdout:(115/127): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-25T19:44:05.108 INFO:teuthology.orchestra.run.smithi119.stdout:(116/127): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-08-25T19:44:05.142 INFO:teuthology.orchestra.run.smithi119.stdout:(117/127): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-08-25T19:44:05.167 INFO:teuthology.orchestra.run.smithi119.stdout:(118/127): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-08-25T19:44:05.234 INFO:teuthology.orchestra.run.smithi119.stdout:(119/127): libarrow-9.0.0-11.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-08-25T19:44:05.259 INFO:teuthology.orchestra.run.smithi119.stdout:(120/127): python3-websocket-client-1.2.3-2.el9 977 kB/s | 90 kB 00:00 2024-08-25T19:44:05.285 INFO:teuthology.orchestra.run.smithi119.stdout:(121/127): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-08-25T19:44:05.301 INFO:teuthology.orchestra.run.smithi119.stdout:(122/127): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-08-25T19:44:05.335 INFO:teuthology.orchestra.run.smithi119.stdout:(123/127): re2-20211101-20.el9.x86_64.rpm 3.8 MB/s | 191 kB 00:00 2024-08-25T19:44:05.360 INFO:teuthology.orchestra.run.smithi119.stdout:(124/127): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-25T19:44:05.419 INFO:teuthology.orchestra.run.smithi119.stdout:(125/127): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-08-25T19:44:05.628 INFO:teuthology.orchestra.run.smithi119.stdout:(126/127): librbd1-19.0.0-5971.gdb4959e4.el9.x8 12 MB/s | 3.2 MB 00:00 2024-08-25T19:44:05.686 INFO:teuthology.orchestra.run.smithi119.stdout:(127/127): librados2-19.0.0-5971.gdb4959e4.el9. 9.9 MB/s | 3.5 MB 00:00 2024-08-25T19:44:05.692 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-08-25T19:44:05.692 INFO:teuthology.orchestra.run.smithi119.stdout:Total 22 MB/s | 215 MB 00:09 2024-08-25T19:44:07.021 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-08-25T19:44:07.146 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-08-25T19:44:07.147 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-08-25T19:44:09.285 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-08-25T19:44:09.285 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-08-25T19:44:09.589 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-08-25T19:44:09.688 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/129 2024-08-25T19:44:09.736 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/129 2024-08-25T19:44:09.817 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/129 2024-08-25T19:44:10.199 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/129 2024-08-25T19:44:10.296 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:19.0.0-5971.gdb4959e4.el9.x86_64 5/129 2024-08-25T19:44:10.474 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:19.0.0-5971.gdb4959e4.el9.x86_64 5/129 2024-08-25T19:44:10.521 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:19.0.0-5971.gdb4959e4.el9.x86_64 6/129 2024-08-25T19:44:10.676 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:19.0.0-5971.gdb4959e4.el9.x86_64 6/129 2024-08-25T19:44:10.729 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:19.0.0-5971.gdb4959e4.el9.x86_64 7/129 2024-08-25T19:44:10.760 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/129 2024-08-25T19:44:10.811 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/129 2024-08-25T19:44:10.841 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:19.0.0-5971.gdb4959e4.el9.x86_64 10/129 2024-08-25T19:44:11.025 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5971.gdb4959e4.el9.x86_64 10/129 2024-08-25T19:44:11.050 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:19.0.0-5971.gdb4959e4.el9.x86 11/129 2024-08-25T19:44:11.167 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5971.gdb4959e4.el9.x86 11/129 2024-08-25T19:44:11.193 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 12/129 2024-08-25T19:44:11.235 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 12/129 2024-08-25T19:44:11.721 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 13/129 2024-08-25T19:44:11.778 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 13/129 2024-08-25T19:44:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : re2-1:20211101-20.el9.x86_64 14/129 2024-08-25T19:44:12.003 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 15/129 2024-08-25T19:44:12.048 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.7-2.el9.x86_64 16/129 2024-08-25T19:44:12.132 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 17/129 2024-08-25T19:44:12.193 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 18/129 2024-08-25T19:44:12.231 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-packaging-20.9-5.el9.noarch 19/129 2024-08-25T19:44:12.284 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 20/129 2024-08-25T19:44:12.335 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 21/129 2024-08-25T19:44:12.366 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-5.4.4-4.el9.x86_64 22/129 2024-08-25T19:44:12.455 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 23/129 2024-08-25T19:44:12.518 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 24/129 2024-08-25T19:44:12.570 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 25/129 2024-08-25T19:44:12.616 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 26/129 2024-08-25T19:44:12.698 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 27/129 2024-08-25T19:44:12.732 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:19.0.0-5971.gdb4959e4.el9. 28/129 2024-08-25T19:44:12.767 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:19.0.0-5971.gdb4959e4.el 29/129 2024-08-25T19:44:12.787 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:19.0.0-5971.gdb4959e4.el9.x86_6 30/129 2024-08-25T19:44:12.870 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 31/129 2024-08-25T19:44:12.923 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : luarocks-3.9.2-5.el9.noarch 32/129 2024-08-25T19:44:13.015 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 33/129 2024-08-25T19:44:13.175 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 34/129 2024-08-25T19:44:13.234 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 35/129 2024-08-25T19:44:13.868 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.9-2.el9.noarch 36/129 2024-08-25T19:44:13.969 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 37/129 2024-08-25T19:44:14.004 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 38/129 2024-08-25T19:44:14.041 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 39/129 2024-08-25T19:44:14.069 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:19.0.0-5971.gdb4959e4.el9.x86_6 40/129 2024-08-25T19:44:14.102 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 41/129 2024-08-25T19:44:14.208 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 42/129 2024-08-25T19:44:14.289 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 43/129 2024-08-25T19:44:14.372 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 44/129 2024-08-25T19:44:14.413 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 45/129 2024-08-25T19:44:14.450 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 46/129 2024-08-25T19:44:14.508 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 47/129 2024-08-25T19:44:14.560 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 48/129 2024-08-25T19:44:14.610 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 49/129 2024-08-25T19:44:14.785 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 50/129 2024-08-25T19:44:14.829 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-08-25T19:44:14.908 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/129 2024-08-25T19:44:14.940 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/129 2024-08-25T19:44:15.021 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/129 2024-08-25T19:44:15.077 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 51/129 2024-08-25T19:44:15.120 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 52/129 2024-08-25T19:44:15.246 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 53/129 2024-08-25T19:44:15.293 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 54/129 2024-08-25T19:44:15.337 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 55/129 2024-08-25T19:44:15.365 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 56/129 2024-08-25T19:44:15.390 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 57/129 2024-08-25T19:44:15.404 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/129 2024-08-25T19:44:15.517 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 58/129 2024-08-25T19:44:15.532 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:19.0.0-5971.gdb4959e4.el9.x86_64 5/129 2024-08-25T19:44:15.669 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:19.0.0-5971.gdb4959e4.el9.x86_64 5/129 2024-08-25T19:44:15.674 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 59/129 2024-08-25T19:44:15.699 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:19.0.0-5971.gdb4959e4.el9.x86_64 6/129 2024-08-25T19:44:15.726 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 60/129 2024-08-25T19:44:15.777 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 61/129 2024-08-25T19:44:15.814 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:19.0.0-5971.gdb4959e4.el9.x86_64 6/129 2024-08-25T19:44:15.823 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 62/129 2024-08-25T19:44:15.860 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 63/129 2024-08-25T19:44:15.870 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:19.0.0-5971.gdb4959e4.el9.x86_64 7/129 2024-08-25T19:44:15.895 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 64/129 2024-08-25T19:44:15.905 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/129 2024-08-25T19:44:15.928 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 65/129 2024-08-25T19:44:15.948 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/129 2024-08-25T19:44:15.971 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 66/129 2024-08-25T19:44:15.977 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:19.0.0-5971.gdb4959e4.el9.x86_64 10/129 2024-08-25T19:44:16.004 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 67/129 2024-08-25T19:44:16.094 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5971.gdb4959e4.el9.x86_64 10/129 2024-08-25T19:44:16.103 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 68/129 2024-08-25T19:44:16.120 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:19.0.0-5971.gdb4959e4.el9.x86 11/129 2024-08-25T19:44:16.163 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 69/129 2024-08-25T19:44:16.199 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 70/129 2024-08-25T19:44:16.201 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5971.gdb4959e4.el9.x86 11/129 2024-08-25T19:44:16.229 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 12/129 2024-08-25T19:44:16.273 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 12/129 2024-08-25T19:44:16.322 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 71/129 2024-08-25T19:44:16.395 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 72/129 2024-08-25T19:44:16.791 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 13/129 2024-08-25T19:44:16.823 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 13/129 2024-08-25T19:44:16.954 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-20.el9.x86_64 14/129 2024-08-25T19:44:16.987 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 15/129 2024-08-25T19:44:17.031 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 16/129 2024-08-25T19:44:17.113 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 17/129 2024-08-25T19:44:17.168 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 18/129 2024-08-25T19:44:17.206 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-packaging-20.9-5.el9.noarch 19/129 2024-08-25T19:44:17.260 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 20/129 2024-08-25T19:44:17.283 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/129 2024-08-25T19:44:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 21/129 2024-08-25T19:44:17.339 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 74/129 2024-08-25T19:44:17.372 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-5.4.4-4.el9.x86_64 22/129 2024-08-25T19:44:17.373 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 75/129 2024-08-25T19:44:17.465 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 23/129 2024-08-25T19:44:17.487 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 76/129 2024-08-25T19:44:17.527 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 24/129 2024-08-25T19:44:17.531 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-devel-3.9.19-7.el9.x86_64 77/129 2024-08-25T19:44:17.562 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 25/129 2024-08-25T19:44:17.608 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 26/129 2024-08-25T19:44:17.690 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 27/129 2024-08-25T19:44:17.724 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:19.0.0-5971.gdb4959e4.el9. 28/129 2024-08-25T19:44:17.764 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.26-1.el9.x86_64 78/129 2024-08-25T19:44:17.767 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:19.0.0-5971.gdb4959e4.el 29/129 2024-08-25T19:44:17.796 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:19.0.0-5971.gdb4959e4.el9.x86_6 30/129 2024-08-25T19:44:17.802 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-openmp-0.3.26-1.el9.x86_64 79/129 2024-08-25T19:44:17.888 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 31/129 2024-08-25T19:44:17.899 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 80/129 2024-08-25T19:44:17.940 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : luarocks-3.9.2-5.el9.noarch 32/129 2024-08-25T19:44:18.032 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 33/129 2024-08-25T19:44:18.192 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 34/129 2024-08-25T19:44:18.261 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 35/129 2024-08-25T19:44:18.719 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 81/129 2024-08-25T19:44:18.867 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 82/129 2024-08-25T19:44:18.901 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 36/129 2024-08-25T19:44:19.011 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 37/129 2024-08-25T19:44:19.046 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 38/129 2024-08-25T19:44:19.091 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 39/129 2024-08-25T19:44:19.129 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:19.0.0-5971.gdb4959e4.el9.x86_6 40/129 2024-08-25T19:44:19.170 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 41/129 2024-08-25T19:44:19.278 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 42/129 2024-08-25T19:44:19.357 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 43/129 2024-08-25T19:44:19.441 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 44/129 2024-08-25T19:44:19.480 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 45/129 2024-08-25T19:44:19.517 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 46/129 2024-08-25T19:44:19.584 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 47/129 2024-08-25T19:44:19.636 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 48/129 2024-08-25T19:44:19.702 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 49/129 2024-08-25T19:44:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 50/129 2024-08-25T19:44:19.927 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 51/129 2024-08-25T19:44:19.971 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 52/129 2024-08-25T19:44:20.097 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 53/129 2024-08-25T19:44:20.140 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 54/129 2024-08-25T19:44:20.322 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 55/129 2024-08-25T19:44:20.350 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 56/129 2024-08-25T19:44:20.366 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 57/129 2024-08-25T19:44:20.461 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/129 2024-08-25T19:44:20.484 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 58/129 2024-08-25T19:44:20.560 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 84/129 2024-08-25T19:44:20.642 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 59/129 2024-08-25T19:44:20.692 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 60/129 2024-08-25T19:44:20.752 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 61/129 2024-08-25T19:44:20.791 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 62/129 2024-08-25T19:44:20.828 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 63/129 2024-08-25T19:44:20.863 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 64/129 2024-08-25T19:44:20.896 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 65/129 2024-08-25T19:44:20.905 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 85/129 2024-08-25T19:44:20.939 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 66/129 2024-08-25T19:44:20.954 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:19.0.0-5971.gdb4959e4.el9.x86_64 86/129 2024-08-25T19:44:20.980 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 67/129 2024-08-25T19:44:21.081 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 68/129 2024-08-25T19:44:21.115 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:19.0.0-5971.gdb4959e4.el9.x86_64 86/129 2024-08-25T19:44:21.131 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 69/129 2024-08-25T19:44:21.143 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:19.0.0-5971.gdb4959e4.el9.x86_64 87/129 2024-08-25T19:44:21.167 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 70/129 2024-08-25T19:44:21.191 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 88/129 2024-08-25T19:44:21.291 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 71/129 2024-08-25T19:44:21.355 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 72/129 2024-08-25T19:44:21.731 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 89/129 2024-08-25T19:44:21.923 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:19.0.0-5971.gdb4959e4.el9.x86_64 90/129 2024-08-25T19:44:22.031 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:19.0.0-5971.gdb4959e4.el9.x86_64 90/129 2024-08-25T19:44:22.058 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 91/129 2024-08-25T19:44:22.250 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/129 2024-08-25T19:44:22.307 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 74/129 2024-08-25T19:44:22.341 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 75/129 2024-08-25T19:44:22.461 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 76/129 2024-08-25T19:44:22.507 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.19-7.el9.x86_64 77/129 2024-08-25T19:44:22.740 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.26-1.el9.x86_64 78/129 2024-08-25T19:44:22.771 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.26-1.el9.x86_64 79/129 2024-08-25T19:44:22.882 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 80/129 2024-08-25T19:44:23.698 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 81/129 2024-08-25T19:44:23.844 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 82/129 2024-08-25T19:44:24.747 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.0.0-5971.gdb4959e4.el9.x86_64 92/129 2024-08-25T19:44:24.780 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:19.0.0-5971.gdb4959e4.el9.x86_64 92/129 2024-08-25T19:44:25.425 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.0.0-5971.gdb4959e4.el9.x86_64 92/129 2024-08-25T19:44:25.475 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:19.0.0-5971.gdb4959e4.el9.x86_64 93/129 2024-08-25T19:44:25.492 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/129 2024-08-25T19:44:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:19.0.0-5971.gdb4959e4.el9.x86_64 93/129 2024-08-25T19:44:25.607 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-25T19:44:25.607 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-25T19:44:25.607 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:44:25.646 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 84/129 2024-08-25T19:44:25.651 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:19.0.0-5971.gdb4959e4.el9.x86_64 94/129 2024-08-25T19:44:25.992 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 85/129 2024-08-25T19:44:26.048 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:19.0.0-5971.gdb4959e4.el9.x86_64 86/129 2024-08-25T19:44:26.168 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:19.0.0-5971.gdb4959e4.el9.x86_64 86/129 2024-08-25T19:44:26.204 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:19.0.0-5971.gdb4959e4.el9.x86_64 87/129 2024-08-25T19:44:26.252 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 88/129 2024-08-25T19:44:26.792 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 89/129 2024-08-25T19:44:26.825 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:19.0.0-5971.gdb4959e4.el9.x86_64 90/129 2024-08-25T19:44:26.916 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:19.0.0-5971.gdb4959e4.el9.x86_64 90/129 2024-08-25T19:44:26.944 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 91/129 2024-08-25T19:44:29.536 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-5971.gdb4959e4.el9.x86_64 92/129 2024-08-25T19:44:29.582 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:19.0.0-5971.gdb4959e4.el9.x86_64 92/129 2024-08-25T19:44:30.223 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-5971.gdb4959e4.el9.x86_64 92/129 2024-08-25T19:44:30.269 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:19.0.0-5971.gdb4959e4.el9.x86_64 93/129 2024-08-25T19:44:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:19.0.0-5971.gdb4959e4.el9.x86_64 93/129 2024-08-25T19:44:30.402 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-25T19:44:30.402 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-25T19:44:30.402 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:44:30.485 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:19.0.0-5971.gdb4959e4.el9.x86_64 94/129 2024-08-25T19:44:37.773 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5971.gdb4959e4.el9.x86_64 94/129 2024-08-25T19:44:37.773 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2024-08-25T19:44:37.774 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2024-08-25T19:44:37.774 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2024-08-25T19:44:37.774 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2024-08-25T19:44:37.774 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2024-08-25T19:44:37.774 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /root 2024-08-25T19:44:37.774 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2024-08-25T19:44:37.774 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:44:37.980 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-volume-2:19.0.0-5971.gdb4959e4.el9.noarch 95/129 2024-08-25T19:44:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-volume-2:19.0.0-5971.gdb4959e4.el9.noarch 95/129 2024-08-25T19:44:38.016 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:38.016 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-25T19:44:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:44:39.590 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:19.0.0-5971.gdb4959e4.el9.x86_64 96/129 2024-08-25T19:44:39.648 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:19.0.0-5971.gdb4959e4.el9.x86_64 96/129 2024-08-25T19:44:39.648 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:39.648 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-25T19:44:39.648 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-25T19:44:39.648 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-25T19:44:39.648 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:44:39.890 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:19.0.0-5971.gdb4959e4.el9.x86_64 97/129 2024-08-25T19:44:39.942 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:19.0.0-5971.gdb4959e4.el9.x86_64 97/129 2024-08-25T19:44:39.942 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:39.942 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-25T19:44:39.942 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-25T19:44:39.942 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-25T19:44:39.942 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:44:40.441 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:19.0.0-5971.gdb4959e4.el9.x86_64 98/129 2024-08-25T19:44:40.491 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:19.0.0-5971.gdb4959e4.el9.x86_64 98/129 2024-08-25T19:44:40.492 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:40.492 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-25T19:44:40.492 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-25T19:44:40.492 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-25T19:44:40.492 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:44:40.592 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ply-3.11-14.el9.noarch 99/129 2024-08-25T19:44:40.656 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 100/129 2024-08-25T19:44:40.881 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 101/129 2024-08-25T19:44:40.944 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 102/129 2024-08-25T19:44:41.030 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 103/129 2024-08-25T19:44:41.136 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 104/129 2024-08-25T19:44:41.293 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 105/129 2024-08-25T19:44:41.344 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 106/129 2024-08-25T19:44:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 107/129 2024-08-25T19:44:41.417 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.9-16.el9.x86_64 108/129 2024-08-25T19:44:41.933 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:19.0.0-5971.gdb4959e4.el9.noarch 109/129 2024-08-25T19:44:42.009 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:19.0.0-5971.gdb4959e4.el9.noarch 109/129 2024-08-25T19:44:42.068 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5971.gdb4959e4.e 110/129 2024-08-25T19:44:42.148 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5971.gdb4959e4. 111/129 2024-08-25T19:44:42.181 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5971.gdb4959e4.el9.noa 112/129 2024-08-25T19:44:42.866 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5971.gdb4959e4.el9.x86_64 94/129 2024-08-25T19:44:42.866 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2024-08-25T19:44:42.866 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2024-08-25T19:44:42.866 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2024-08-25T19:44:42.867 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2024-08-25T19:44:42.867 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2024-08-25T19:44:42.867 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2024-08-25T19:44:42.867 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2024-08-25T19:44:42.867 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:44:43.071 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:19.0.0-5971.gdb4959e4.el9.noarch 95/129 2024-08-25T19:44:43.107 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:19.0.0-5971.gdb4959e4.el9.noarch 95/129 2024-08-25T19:44:43.107 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:43.107 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-25T19:44:43.107 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:44:44.673 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:19.0.0-5971.gdb4959e4.el9.x86_64 96/129 2024-08-25T19:44:44.731 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:19.0.0-5971.gdb4959e4.el9.x86_64 96/129 2024-08-25T19:44:44.731 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:44.731 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-25T19:44:44.731 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-25T19:44:44.732 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-25T19:44:44.732 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:44:44.974 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:19.0.0-5971.gdb4959e4.el9.x86_64 97/129 2024-08-25T19:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:19.0.0-5971.gdb4959e4.el9.x86_64 97/129 2024-08-25T19:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-25T19:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-25T19:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-25T19:44:45.025 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:44:45.524 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:19.0.0-5971.gdb4959e4.el9.x86_64 98/129 2024-08-25T19:44:45.575 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:19.0.0-5971.gdb4959e4.el9.x86_64 98/129 2024-08-25T19:44:45.575 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:45.575 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-25T19:44:45.575 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-25T19:44:45.576 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-25T19:44:45.576 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:44:45.650 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 99/129 2024-08-25T19:44:45.722 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 100/129 2024-08-25T19:44:45.755 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5971.gdb4959e4.el9.noa 112/129 2024-08-25T19:44:45.829 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5971.gdb4959e4.el9.n 113/129 2024-08-25T19:44:45.948 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 101/129 2024-08-25T19:44:46.020 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 102/129 2024-08-25T19:44:46.113 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 103/129 2024-08-25T19:44:46.223 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 104/129 2024-08-25T19:44:46.387 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 105/129 2024-08-25T19:44:46.454 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 106/129 2024-08-25T19:44:46.492 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 107/129 2024-08-25T19:44:46.518 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-16.el9.x86_64 108/129 2024-08-25T19:44:46.913 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5971.gdb4959e4.el9.n 113/129 2024-08-25T19:44:46.945 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5971.gdb4 114/129 2024-08-25T19:44:47.025 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:19.0.0-5971.gdb4959e4.el9.noarch 109/129 2024-08-25T19:44:47.059 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:19.0.0-5971.gdb4959e4.el9.noarch 109/129 2024-08-25T19:44:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5971.gdb4 114/129 2024-08-25T19:44:47.101 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5971.gdb4959e4.e 110/129 2024-08-25T19:44:47.182 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5971.gdb4959e4. 111/129 2024-08-25T19:44:47.215 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5971.gdb4959e4.el9.noa 112/129 2024-08-25T19:44:47.277 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5971.gdb4959e4.el 115/129 2024-08-25T19:44:47.306 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:19.0.0-5971.gdb4959e4.el9.x86_64 116/129 2024-08-25T19:44:47.357 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5971.gdb4959e4.el9.x86_64 116/129 2024-08-25T19:44:47.358 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:47.358 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-25T19:44:47.358 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-25T19:44:47.358 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-25T19:44:47.358 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:44:47.415 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:19.0.0-5971.gdb4959e4.el9.noarch 117/129 2024-08-25T19:44:47.441 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5971.gdb4959e4.el9.noarch 117/129 2024-08-25T19:44:47.550 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:19.0.0-5971.gdb4959e4.el9.x86_64 118/129 2024-08-25T19:44:48.580 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:19.0.0-5971.gdb4959e4.el9.x86_64 119/129 2024-08-25T19:44:48.608 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 120/129 2024-08-25T19:44:48.659 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 120/129 2024-08-25T19:44:48.659 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:48.659 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-25T19:44:48.659 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-25T19:44:48.659 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-25T19:44:48.659 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:44:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5971.gdb495 121/129 2024-08-25T19:44:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5971.gdb495 121/129 2024-08-25T19:44:48.767 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:48.767 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-25T19:44:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:44:49.118 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:19.0.0-5971.gdb4959e4.el9.x86_64 122/129 2024-08-25T19:44:49.170 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5971.gdb4959e4.el9.x86_64 122/129 2024-08-25T19:44:49.170 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:49.170 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-25T19:44:49.170 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-25T19:44:49.170 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-25T19:44:49.170 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:44:50.789 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5971.gdb4959e4.el9.noa 112/129 2024-08-25T19:44:50.847 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5971.gdb4959e4.el9.n 113/129 2024-08-25T19:44:51.932 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5971.gdb4959e4.el9.n 113/129 2024-08-25T19:44:51.963 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5971.gdb4 114/129 2024-08-25T19:44:52.114 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5971.gdb4 114/129 2024-08-25T19:44:52.295 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5971.gdb4959e4.el 115/129 2024-08-25T19:44:52.323 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:19.0.0-5971.gdb4959e4.el9.x86_64 116/129 2024-08-25T19:44:52.375 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5971.gdb4959e4.el9.x86_64 116/129 2024-08-25T19:44:52.375 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:52.375 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-25T19:44:52.375 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-25T19:44:52.375 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-25T19:44:52.375 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:44:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:19.0.0-5971.gdb4959e4.el9.noarch 117/129 2024-08-25T19:44:52.459 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5971.gdb4959e4.el9.noarch 117/129 2024-08-25T19:44:52.565 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:19.0.0-5971.gdb4959e4.el9.x86_64 118/129 2024-08-25T19:44:53.592 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:19.0.0-5971.gdb4959e4.el9.x86_64 119/129 2024-08-25T19:44:53.635 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 120/129 2024-08-25T19:44:53.686 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 120/129 2024-08-25T19:44:53.686 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:53.686 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-25T19:44:53.686 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-25T19:44:53.686 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-25T19:44:53.686 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:44:53.735 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5971.gdb495 121/129 2024-08-25T19:44:53.784 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5971.gdb495 121/129 2024-08-25T19:44:53.784 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:53.784 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-25T19:44:53.784 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:44:54.136 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:19.0.0-5971.gdb4959e4.el9.x86_64 122/129 2024-08-25T19:44:54.187 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5971.gdb4959e4.el9.x86_64 122/129 2024-08-25T19:44:54.187 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-25T19:44:54.187 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-25T19:44:54.187 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-25T19:44:54.187 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-25T19:44:54.187 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:44:54.248 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:19.0.0-5971.gdb4959e4.el9.x86_64 123/129 2024-08-25T19:44:54.289 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:19.0.0-5971.gdb4959e4.el9.x86_64 124/129 2024-08-25T19:44:54.321 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:19.0.0-5971.gdb4959e4.el9.x86_64 125/129 2024-08-25T19:44:54.373 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:19.0.0-5971.gdb4959e4.el9.x86_ 126/129 2024-08-25T19:44:54.404 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : pv-1.6.20-1.el9.x86_64 127/129 2024-08-25T19:44:54.404 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 128/129 2024-08-25T19:44:54.511 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 128/129 2024-08-25T19:44:54.512 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 129/129 2024-08-25T19:44:54.899 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 129/129 2024-08-25T19:44:54.899 INFO:teuthology.orchestra.run.smithi121.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-08-25T19:44:54.899 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:44:59.259 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:19.0.0-5971.gdb4959e4.el9.x86_64 123/129 2024-08-25T19:44:59.300 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:19.0.0-5971.gdb4959e4.el9.x86_64 124/129 2024-08-25T19:44:59.331 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:19.0.0-5971.gdb4959e4.el9.x86_64 125/129 2024-08-25T19:44:59.381 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:19.0.0-5971.gdb4959e4.el9.x86_ 126/129 2024-08-25T19:44:59.414 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : pv-1.6.20-1.el9.x86_64 127/129 2024-08-25T19:44:59.414 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 128/129 2024-08-25T19:44:59.538 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 128/129 2024-08-25T19:44:59.539 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 129/129 2024-08-25T19:44:59.933 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 129/129 2024-08-25T19:44:59.934 INFO:teuthology.orchestra.run.smithi119.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-08-25T19:44:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:00.490 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:19.0.0-5971.gdb4959e4.el9.x86_64 1/129 2024-08-25T19:45:00.490 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:19.0.0-5971.gdb4959e4.el9.x86_64 2/129 2024-08-25T19:45:00.490 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:19.0.0-5971.gdb4959e4.el9.x86_64 3/129 2024-08-25T19:45:00.490 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:19.0.0-5971.gdb4959e4.el9.x86_64 4/129 2024-08-25T19:45:00.490 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5971.gdb495 5/129 2024-08-25T19:45:00.490 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:19.0.0-5971.gdb4959e4.el9.x86_64 6/129 2024-08-25T19:45:00.490 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:19.0.0-5971.gdb4959e4.el9.x86_64 7/129 2024-08-25T19:45:00.490 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:19.0.0-5971.gdb4959e4.el9.x86_64 8/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:19.0.0-5971.gdb4959e4.el9.x86_64 9/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 10/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:19.0.0-5971.gdb4959e4.el9.x86_64 11/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:19.0.0-5971.gdb4959e4.el9.x86_64 12/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:19.0.0-5971.gdb4959e4.el9.x86_ 13/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:19.0.0-5971.gdb4959e4.el9.x86_64 14/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:19.0.0-5971.gdb4959e4.el9.x86_64 15/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:19.0.0-5971.gdb4959e4.el9.x86_6 16/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:19.0.0-5971.gdb4959e4.el9.x86 17/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:19.0.0-5971.gdb4959e4.el9.x86_64 18/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5971.gdb4959e4.el 19/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:19.0.0-5971.gdb4959e4.el9. 20/129 2024-08-25T19:45:00.491 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:19.0.0-5971.gdb4959e4.el9.x86_6 21/129 2024-08-25T19:45:00.492 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:19.0.0-5971.gdb4959e4.el9.x86_64 22/129 2024-08-25T19:45:00.492 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:19.0.0-5971.gdb4959e4.el9.x86_64 23/129 2024-08-25T19:45:00.492 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 24/129 2024-08-25T19:45:00.492 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:19.0.0-5971.gdb4959e4.el9.x86_64 25/129 2024-08-25T19:45:00.501 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:19.0.0-5971.gdb4959e4.el9.x86_64 26/129 2024-08-25T19:45:00.501 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:19.0.0-5971.gdb4959e4.el9.x86_64 27/129 2024-08-25T19:45:00.501 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5971.gdb4959e4. 28/129 2024-08-25T19:45:00.501 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5971.gdb4959e4.el9.noa 29/129 2024-08-25T19:45:00.501 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5971.gdb4959e4.el9.n 30/129 2024-08-25T19:45:00.501 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5971.gdb4 31/129 2024-08-25T19:45:00.501 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5971.gdb4959e4.el 32/129 2024-08-25T19:45:00.501 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5971.gdb4959e4.el9.noarch 33/129 2024-08-25T19:45:00.501 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5971.gdb4959e4.e 34/129 2024-08-25T19:45:00.501 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-volume-2:19.0.0-5971.gdb4959e4.el9.noarch 35/129 2024-08-25T19:45:00.501 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:19.0.0-5971.gdb4959e4.el9.noarch 36/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 37/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 38/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 39/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 45/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 46/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 47/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 48/129 2024-08-25T19:45:00.502 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 49/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 50/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 51/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 52/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.26-1.el9.x86_64 55/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-openmp-0.3.26-1.el9.x86_64 56/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 57/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-devel-3.9.19-7.el9.x86_64 59/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 60/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/129 2024-08-25T19:45:00.503 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/129 2024-08-25T19:45:00.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/129 2024-08-25T19:45:00.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/129 2024-08-25T19:45:00.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/129 2024-08-25T19:45:00.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/129 2024-08-25T19:45:00.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/129 2024-08-25T19:45:00.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/129 2024-08-25T19:45:00.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/129 2024-08-25T19:45:00.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/129 2024-08-25T19:45:00.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/129 2024-08-25T19:45:00.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/129 2024-08-25T19:45:00.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/129 2024-08-25T19:45:00.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 77/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 78/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 79/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 80/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 81/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 82/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 83/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 84/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 85/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 86/129 2024-08-25T19:45:00.505 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 87/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 88/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 89/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 90/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 91/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 92/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : pv-1.6.20-1.el9.x86_64 93/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 94/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 95/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 96/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 97/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 98/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 99/129 2024-08-25T19:45:00.506 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 100/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 101/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 102/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 103/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 104/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 105/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 106/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 107/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 108/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 109/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 110/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 111/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 112/129 2024-08-25T19:45:00.507 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 113/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 114/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 115/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 116/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 117/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 118/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 119/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 120/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 121/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 122/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 123/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : re2-1:20211101-20.el9.x86_64 124/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 125/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:19.0.0-5971.gdb4959e4.el9.x86_64 126/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 127/129 2024-08-25T19:45:00.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:19.0.0-5971.gdb4959e4.el9.x86_64 128/129 2024-08-25T19:45:02.634 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 129/129 2024-08-25T19:45:02.634 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.635 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:02.636 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:02.637 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-25T19:45:02.637 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-25T19:45:02.637 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-25T19:45:02.637 INFO:teuthology.orchestra.run.smithi121.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-25T19:45:02.637 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-25T19:45:02.637 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-25T19:45:02.637 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-25T19:45:02.637 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-25T19:45:02.637 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.637 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.637 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.637 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: lua-5.4.4-4.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.26-1.el9.x86_64 2024-08-25T19:45:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp-0.3.26-1.el9.x86_64 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: pv-1.6.20-1.el9.x86_64 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-25T19:45:02.639 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel-3.9.19-7.el9.x86_64 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-25T19:45:02.640 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply-3.11-14.el9.noarch 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-25T19:45:02.641 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-25T19:45:02.642 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: re2-1:20211101-20.el9.x86_64 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-25T19:45:02.643 INFO:teuthology.orchestra.run.smithi121.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-25T19:45:02.644 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-25T19:45:02.644 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:02.644 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-08-25T19:45:03.144 DEBUG:teuthology.parallel:result is None 2024-08-25T19:45:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:19.0.0-5971.gdb4959e4.el9.x86_64 1/129 2024-08-25T19:45:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:19.0.0-5971.gdb4959e4.el9.x86_64 2/129 2024-08-25T19:45:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:19.0.0-5971.gdb4959e4.el9.x86_64 3/129 2024-08-25T19:45:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:19.0.0-5971.gdb4959e4.el9.x86_64 4/129 2024-08-25T19:45:05.752 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5971.gdb495 5/129 2024-08-25T19:45:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:19.0.0-5971.gdb4959e4.el9.x86_64 6/129 2024-08-25T19:45:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:19.0.0-5971.gdb4959e4.el9.x86_64 7/129 2024-08-25T19:45:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:19.0.0-5971.gdb4959e4.el9.x86_64 8/129 2024-08-25T19:45:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:19.0.0-5971.gdb4959e4.el9.x86_64 9/129 2024-08-25T19:45:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 10/129 2024-08-25T19:45:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:19.0.0-5971.gdb4959e4.el9.x86_64 11/129 2024-08-25T19:45:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:19.0.0-5971.gdb4959e4.el9.x86_64 12/129 2024-08-25T19:45:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:19.0.0-5971.gdb4959e4.el9.x86_ 13/129 2024-08-25T19:45:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:19.0.0-5971.gdb4959e4.el9.x86_64 14/129 2024-08-25T19:45:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:19.0.0-5971.gdb4959e4.el9.x86_64 15/129 2024-08-25T19:45:05.753 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:19.0.0-5971.gdb4959e4.el9.x86_6 16/129 2024-08-25T19:45:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:19.0.0-5971.gdb4959e4.el9.x86 17/129 2024-08-25T19:45:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:19.0.0-5971.gdb4959e4.el9.x86_64 18/129 2024-08-25T19:45:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5971.gdb4959e4.el 19/129 2024-08-25T19:45:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:19.0.0-5971.gdb4959e4.el9. 20/129 2024-08-25T19:45:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:19.0.0-5971.gdb4959e4.el9.x86_6 21/129 2024-08-25T19:45:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:19.0.0-5971.gdb4959e4.el9.x86_64 22/129 2024-08-25T19:45:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:19.0.0-5971.gdb4959e4.el9.x86_64 23/129 2024-08-25T19:45:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 24/129 2024-08-25T19:45:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:19.0.0-5971.gdb4959e4.el9.x86_64 25/129 2024-08-25T19:45:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:19.0.0-5971.gdb4959e4.el9.x86_64 26/129 2024-08-25T19:45:05.754 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:19.0.0-5971.gdb4959e4.el9.x86_64 27/129 2024-08-25T19:45:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5971.gdb4959e4. 28/129 2024-08-25T19:45:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5971.gdb4959e4.el9.noa 29/129 2024-08-25T19:45:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5971.gdb4959e4.el9.n 30/129 2024-08-25T19:45:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5971.gdb4 31/129 2024-08-25T19:45:05.765 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5971.gdb4959e4.el 32/129 2024-08-25T19:45:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5971.gdb4959e4.el9.noarch 33/129 2024-08-25T19:45:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5971.gdb4959e4.e 34/129 2024-08-25T19:45:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:19.0.0-5971.gdb4959e4.el9.noarch 35/129 2024-08-25T19:45:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:19.0.0-5971.gdb4959e4.el9.noarch 36/129 2024-08-25T19:45:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 37/129 2024-08-25T19:45:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 38/129 2024-08-25T19:45:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 39/129 2024-08-25T19:45:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/129 2024-08-25T19:45:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/129 2024-08-25T19:45:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/129 2024-08-25T19:45:05.766 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/129 2024-08-25T19:45:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/129 2024-08-25T19:45:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 45/129 2024-08-25T19:45:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 46/129 2024-08-25T19:45:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 47/129 2024-08-25T19:45:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 48/129 2024-08-25T19:45:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 49/129 2024-08-25T19:45:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 50/129 2024-08-25T19:45:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 51/129 2024-08-25T19:45:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 52/129 2024-08-25T19:45:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/129 2024-08-25T19:45:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/129 2024-08-25T19:45:05.767 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.26-1.el9.x86_64 55/129 2024-08-25T19:45:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.26-1.el9.x86_64 56/129 2024-08-25T19:45:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 57/129 2024-08-25T19:45:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/129 2024-08-25T19:45:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.19-7.el9.x86_64 59/129 2024-08-25T19:45:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 60/129 2024-08-25T19:45:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/129 2024-08-25T19:45:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/129 2024-08-25T19:45:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/129 2024-08-25T19:45:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/129 2024-08-25T19:45:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/129 2024-08-25T19:45:05.768 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/129 2024-08-25T19:45:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/129 2024-08-25T19:45:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/129 2024-08-25T19:45:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/129 2024-08-25T19:45:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/129 2024-08-25T19:45:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/129 2024-08-25T19:45:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/129 2024-08-25T19:45:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/129 2024-08-25T19:45:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/129 2024-08-25T19:45:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/129 2024-08-25T19:45:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/129 2024-08-25T19:45:05.769 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 77/129 2024-08-25T19:45:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 78/129 2024-08-25T19:45:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 79/129 2024-08-25T19:45:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 80/129 2024-08-25T19:45:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 81/129 2024-08-25T19:45:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 82/129 2024-08-25T19:45:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 83/129 2024-08-25T19:45:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 84/129 2024-08-25T19:45:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 85/129 2024-08-25T19:45:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 86/129 2024-08-25T19:45:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 87/129 2024-08-25T19:45:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 88/129 2024-08-25T19:45:05.770 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 89/129 2024-08-25T19:45:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 90/129 2024-08-25T19:45:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 91/129 2024-08-25T19:45:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 92/129 2024-08-25T19:45:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : pv-1.6.20-1.el9.x86_64 93/129 2024-08-25T19:45:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 94/129 2024-08-25T19:45:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 95/129 2024-08-25T19:45:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 96/129 2024-08-25T19:45:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 97/129 2024-08-25T19:45:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 98/129 2024-08-25T19:45:05.771 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 99/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 100/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 101/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 102/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 103/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 104/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 105/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 106/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 107/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 108/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 109/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 110/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 111/129 2024-08-25T19:45:05.772 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 112/129 2024-08-25T19:45:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 113/129 2024-08-25T19:45:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 114/129 2024-08-25T19:45:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 115/129 2024-08-25T19:45:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 116/129 2024-08-25T19:45:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 117/129 2024-08-25T19:45:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 118/129 2024-08-25T19:45:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 119/129 2024-08-25T19:45:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 120/129 2024-08-25T19:45:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 121/129 2024-08-25T19:45:05.773 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 122/129 2024-08-25T19:45:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 123/129 2024-08-25T19:45:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-20.el9.x86_64 124/129 2024-08-25T19:45:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 125/129 2024-08-25T19:45:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:19.0.0-5971.gdb4959e4.el9.x86_64 126/129 2024-08-25T19:45:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 127/129 2024-08-25T19:45:05.774 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:19.0.0-5971.gdb4959e4.el9.x86_64 128/129 2024-08-25T19:45:08.030 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 129/129 2024-08-25T19:45:08.030 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:08.030 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-08-25T19:45:08.030 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.030 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.030 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:08.031 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:19.0.0-5971.gdb4959e4.el9.noarch 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-25T19:45:08.032 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-25T19:45:08.033 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: lua-5.4.4-4.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.26-1.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.26-1.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-25T19:45:08.034 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-25T19:45:08.035 INFO:teuthology.orchestra.run.smithi119.stdout: pv-1.6.20-1.el9.x86_64 2024-08-25T19:45:08.035 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-25T19:45:08.035 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-25T19:45:08.035 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-25T19:45:08.035 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-25T19:45:08.035 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.035 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.035 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.035 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-25T19:45:08.035 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-25T19:45:08.035 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-25T19:45:08.035 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.19-7.el9.x86_64 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-25T19:45:08.036 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-25T19:45:08.037 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-25T19:45:08.037 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-25T19:45:08.037 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-25T19:45:08.037 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-25T19:45:08.037 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-25T19:45:08.037 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-25T19:45:08.037 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-25T19:45:08.037 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-25T19:45:08.037 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-25T19:45:08.037 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-25T19:45:08.037 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2024-08-25T19:45:08.037 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.038 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-25T19:45:08.039 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-25T19:45:08.040 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.040 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.040 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:19.0.0-5971.gdb4959e4.el9.x86_64 2024-08-25T19:45:08.040 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-20.el9.x86_64 2024-08-25T19:45:08.040 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-25T19:45:08.040 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-25T19:45:08.040 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-25T19:45:08.040 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:08.040 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-08-25T19:45:08.546 DEBUG:teuthology.parallel:result is None 2024-08-25T19:45:08.547 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=db4959e44f0252a53b4d8688fbf88b3dda70ea12 2024-08-25T19:45:08.687 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-25T19:45:08.733 INFO:teuthology.orchestra.run.smithi119.stdout:19.0.0-5971.gdb4959e4.el9 2024-08-25T19:45:08.734 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5971.gdb4959e4.el9 2024-08-25T19:45:08.734 INFO:teuthology.task.install:The correct ceph version 19.0.0-5971.gdb4959e4 is installed. 2024-08-25T19:45:08.736 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=db4959e44f0252a53b4d8688fbf88b3dda70ea12 2024-08-25T19:45:08.871 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-25T19:45:08.921 INFO:teuthology.orchestra.run.smithi121.stdout:19.0.0-5971.gdb4959e4.el9 2024-08-25T19:45:08.921 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5971.gdb4959e4.el9 2024-08-25T19:45:08.921 INFO:teuthology.task.install:The correct ceph version 19.0.0-5971.gdb4959e4 is installed. 2024-08-25T19:45:08.923 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-25T19:45:08.923 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:08.923 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-25T19:45:08.981 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:08.981 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-25T19:45:09.042 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-25T19:45:09.042 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:09.042 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-08-25T19:45:09.097 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-25T19:45:09.194 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:09.194 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2024-08-25T19:45:09.251 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-25T19:45:09.346 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-25T19:45:09.347 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:09.347 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-25T19:45:09.402 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-25T19:45:09.497 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:09.497 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-25T19:45:09.552 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-25T19:45:09.646 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-25T19:45:09.646 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:09.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-08-25T19:45:09.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-25T19:45:09.799 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:09.799 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2024-08-25T19:45:09.854 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-25T19:45:09.948 INFO:teuthology.run_tasks:Running task ceph... 2024-08-25T19:45:10.027 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-08-25T19:45:10.028 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2024-08-25T19:45:10.032 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /var/log/ceph 2024-08-25T19:45:10.087 INFO:tasks.ceph:Disabling ceph logrotate... 2024-08-25T19:45:10.087 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-25T19:45:10.091 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-25T19:45:10.145 INFO:tasks.ceph:Creating extra log directories... 2024-08-25T19:45:10.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-25T19:45:10.149 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-25T19:45:10.212 INFO:tasks.ceph:Creating ceph cluster cluster1... 2024-08-25T19:45:10.212 INFO:tasks.ceph:config {'conf': {'client': {'rbd default map options': 'ms_mode=legacy'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'cluster1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-08-25T19:45:10.212 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544', 'branch': 'main', 'description': 'krbd/mirror/{bluestore-bitmap clusters/2-node conf install/ceph ms_mode$/{legacy} tasks/compare-mirror-images}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7870544', 'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '8618c856c9a644ad80ca70c748bebe0c9b6227d4'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': '2 ceph clusters with 1 mon, 1 mgr and 3 osd each'}], 'name': 'teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'rbd default map options': 'ms_mode=legacy'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12'}, 'extra_system_packages': ['pv']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2'], ['cluster2.mon.a', 'cluster2.mgr.x', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster2.client.mirror', 'cluster2.client.mirror.0']], 'seed': 7857, 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12', 'sleep_before_teardown': 0, 'subset': '0/4', 'suite': 'krbd', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_db4959e44f0252a53b4d8688fbf88b3dda70ea12/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12', 'targets': {'smithi119.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLqR8VvVK77NDOLReMDLA0UunAoSi8mErvCIt51FxQdFsTnMvrtmPsn94raf9Z/ZxUUPgLrklOSIjbZn+LeZtSA=', 'smithi121.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFSv2EVnz87750syph3/YS1Efgvmc3yIYvBNKk3Bi1s9Hk1w8k3ZHfg1lD5TTbZYeIWMXihSQDjR7bzAai1KHj8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '8618c856c9a644ad80ca70c748bebe0c9b6227d4'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'flavor': 'default', 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12', 'extra_system_packages': ['pv']}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'rbd default map options': 'ms_mode=legacy'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12'}}, {'ceph': {'cluster': 'cluster2'}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': False}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': False}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/compare_mirror_images.sh']}, 'env': {'RBD_DEVICE_TYPE': 'krbd', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '3h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '4d4c5e30374204f3b32f867e61bb4e1a3880f672', 'timestamp': '2024-08-24_20:48:16', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-08-25T19:45:10.213 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2024-08-25T19:45:10.217 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2024-08-25T19:45:10.251 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2024-08-25T19:45:10.291 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/run/ceph 2024-08-25T19:45:10.346 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:10.346 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-08-25T19:45:10.381 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-25T19:45:10.381 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-08-25T19:45:10.456 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-25T19:45:10.457 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-25T19:45:10.457 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-08-25T19:45:10.457 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-25T19:45:10.457 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-08-25T19:45:10.457 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-08-25 19:45:00.257364750 +0000 2024-08-25T19:45:10.457 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-08-25 19:43:00.848796587 +0000 2024-08-25T19:45:10.457 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-08-25 19:43:00.848796587 +0000 2024-08-25T19:45:10.457 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-08-25 19:43:00.848796587 +0000 2024-08-25T19:45:10.457 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-25T19:45:10.550 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-08-25T19:45:10.550 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-08-25T19:45:10.550 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000314938 s, 1.6 MB/s 2024-08-25T19:45:10.552 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-25T19:45:10.588 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-08-25T19:45:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-25T19:45:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-25T19:45:10.664 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-08-25T19:45:10.664 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-25T19:45:10.664 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-08-25T19:45:10.664 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-08-25 19:45:00.257364750 +0000 2024-08-25T19:45:10.664 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-08-25 19:43:01.308783753 +0000 2024-08-25T19:45:10.664 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-08-25 19:43:01.308783753 +0000 2024-08-25T19:45:10.664 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-08-25 19:43:01.308783753 +0000 2024-08-25T19:45:10.664 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-25T19:45:10.756 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-08-25T19:45:10.756 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-08-25T19:45:10.757 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000268309 s, 1.9 MB/s 2024-08-25T19:45:10.759 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-25T19:45:10.795 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-08-25T19:45:10.869 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-25T19:45:10.870 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-25T19:45:10.870 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-08-25T19:45:10.870 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-25T19:45:10.870 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-08-25T19:45:10.870 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-08-25 19:45:00.258364725 +0000 2024-08-25T19:45:10.870 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-08-25 19:43:01.810769754 +0000 2024-08-25T19:45:10.870 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-08-25 19:43:01.810769754 +0000 2024-08-25T19:45:10.870 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-08-25 19:43:01.810769754 +0000 2024-08-25T19:45:10.870 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-25T19:45:10.961 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-08-25T19:45:10.962 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-08-25T19:45:10.962 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.00026699 s, 1.9 MB/s 2024-08-25T19:45:10.964 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-25T19:45:11.001 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-08-25T19:45:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-25T19:45:11.075 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-25T19:45:11.076 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-08-25T19:45:11.076 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-25T19:45:11.076 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-08-25T19:45:11.076 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-08-25 19:45:00.259364701 +0000 2024-08-25T19:45:11.076 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-08-25 19:43:02.276756760 +0000 2024-08-25T19:45:11.076 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-08-25 19:43:02.276756760 +0000 2024-08-25T19:45:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-08-25 19:43:02.276756760 +0000 2024-08-25T19:45:11.076 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-25T19:45:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-08-25T19:45:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-08-25T19:45:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000480484 s, 1.1 MB/s 2024-08-25T19:45:11.173 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-25T19:45:11.210 INFO:tasks.ceph:osd dev map: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2024-08-25T19:45:11.210 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2024-08-25T19:45:11.210 INFO:tasks.ceph:Generating config... 2024-08-25T19:45:11.212 INFO:tasks.ceph:[client] rbd default map options = ms_mode=legacy 2024-08-25T19:45:11.212 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-08-25T19:45:11.212 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-08-25T19:45:11.212 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-08-25T19:45:11.212 INFO:tasks.ceph:[mgr] debug ms = 1 2024-08-25T19:45:11.212 INFO:tasks.ceph:[mon] debug mon = 20 2024-08-25T19:45:11.212 INFO:tasks.ceph:[mon] debug ms = 1 2024-08-25T19:45:11.212 INFO:tasks.ceph:[mon] debug paxos = 20 2024-08-25T19:45:11.212 INFO:tasks.ceph:[osd] bdev async discard = True 2024-08-25T19:45:11.212 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-08-25T19:45:11.212 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-08-25T19:45:11.212 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-08-25T19:45:11.213 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-08-25T19:45:11.213 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-08-25T19:45:11.213 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-08-25T19:45:11.213 INFO:tasks.ceph:[osd] debug ms = 1 2024-08-25T19:45:11.213 INFO:tasks.ceph:[osd] debug osd = 20 2024-08-25T19:45:11.213 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-08-25T19:45:11.213 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-08-25T19:45:11.213 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-08-25T19:45:11.213 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-08-25T19:45:11.213 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-08-25T19:45:11.213 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-08-25T19:45:11.213 INFO:tasks.ceph:Setting up cluster1.mon.a... 2024-08-25T19:45:11.214 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster1.keyring 2024-08-25T19:45:11.340 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/cluster1.keyring 2024-08-25T19:45:11.346 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster1.keyring 2024-08-25T19:45:11.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2024-08-25T19:45:11.484 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster1.mon.a', '172.21.15.119')] 2024-08-25T19:45:11.485 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.119', 'mon warn on pool no app': False, 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd default map options': 'ms_mode=legacy'}, 'mon.a': {}} 2024-08-25T19:45:11.485 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:11.485 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-08-25T19:45:11.520 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.119 --print /home/ubuntu/cephtest/cluster1.monmap 2024-08-25T19:45:11.630 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster1.monmap 2024-08-25T19:45:11.630 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: generated fsid c5c74996-f4b7-4137-83c8-02707cfa2a6b 2024-08-25T19:45:11.630 INFO:teuthology.orchestra.run.smithi119.stdout:setting min_mon_release = quincy 2024-08-25T19:45:11.630 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 0 2024-08-25T19:45:11.630 INFO:teuthology.orchestra.run.smithi119.stdout:fsid c5c74996-f4b7-4137-83c8-02707cfa2a6b 2024-08-25T19:45:11.630 INFO:teuthology.orchestra.run.smithi119.stdout:last_changed 2024-08-25T19:45:11.630012+0000 2024-08-25T19:45:11.630 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-08-25T19:45:11.630012+0000 2024-08-25T19:45:11.630 INFO:teuthology.orchestra.run.smithi119.stdout:min_mon_release 17 (quincy) 2024-08-25T19:45:11.631 INFO:teuthology.orchestra.run.smithi119.stdout:election_strategy: 1 2024-08-25T19:45:11.631 INFO:teuthology.orchestra.run.smithi119.stdout:0: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.a 2024-08-25T19:45:11.631 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster1.monmap (1 monitors) 2024-08-25T19:45:11.633 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-08-25T19:45:11.667 INFO:tasks.ceph:Writing /etc/ceph/cluster1.conf for FSID c5c74996-f4b7-4137-83c8-02707cfa2a6b... 2024-08-25T19:45:11.668 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2024-08-25T19:45:11.711 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2024-08-25T19:45:11.803 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-08-25T19:45:11.803 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-08-25T19:45:11.803 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-25T19:45:11.803 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-08-25T19:45:11.804 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.804 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-08-25T19:45:11.804 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.804 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-08-25T19:45:11.804 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.804 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-08-25T19:45:11.804 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-08-25T19:45:11.804 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.804 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-08-25T19:45:11.804 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-08-25T19:45:11.804 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.804 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-25T19:45:11.805 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-08-25T19:45:11.805 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.805 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-08-25T19:45:11.805 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-25T19:45:11.805 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.805 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-08-25T19:45:11.805 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.805 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-08-25T19:45:11.805 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-08-25T19:45:11.805 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-08-25T19:45:11.805 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-08-25T19:45:11.805 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-08-25T19:45:11.806 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-08-25T19:45:11.806 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-08-25T19:45:11.806 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-25T19:45:11.806 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-25T19:45:11.806 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-08-25T19:45:11.806 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.806 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-25T19:45:11.806 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.806 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-08-25T19:45:11.806 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.806 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-08-25T19:45:11.806 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.807 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-08-25T19:45:11.807 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-08-25T19:45:11.807 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-08-25T19:45:11.807 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = 172.21.15.119 2024-08-25T19:45:11.807 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on pool no app = False 2024-08-25T19:45:11.807 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on skipped message = False 2024-08-25T19:45:11.807 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = c5c74996-f4b7-4137-83c8-02707cfa2a6b 2024-08-25T19:45:11.807 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.807 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-08-25T19:45:11.807 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-08-25T19:45:11.807 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.807 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-08-25T19:45:11.808 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2024-08-25T19:45:11.808 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-08-25T19:45:11.808 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.808 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-08-25T19:45:11.808 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-08-25T19:45:11.808 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.808 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-08-25T19:45:11.808 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-08-25T19:45:11.808 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-08-25T19:45:11.808 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-08-25T19:45:11.809 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.810 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-08-25T19:45:11.810 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-08-25T19:45:11.810 INFO:teuthology.orchestra.run.smithi119.stdout: bdev async discard = True 2024-08-25T19:45:11.810 INFO:teuthology.orchestra.run.smithi119.stdout: bdev enable discard = True 2024-08-25T19:45:11.810 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore allocator = bitmap 2024-08-25T19:45:11.810 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-08-25T19:45:11.810 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-08-25T19:45:11.810 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 1/20 2024-08-25T19:45:11.810 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 1/20 2024-08-25T19:45:11.810 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-08-25T19:45:11.810 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 4/10 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-08-25T19:45:11.811 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-08-25T19:45:11.812 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-08-25T19:45:11.812 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-08-25T19:45:11.812 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.812 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-08-25T19:45:11.812 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-08-25T19:45:11.812 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-08-25T19:45:11.812 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-08-25T19:45:11.812 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-08-25T19:45:11.812 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-08-25T19:45:11.812 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-08-25T19:45:11.812 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-08-25T19:45:11.812 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-08-25T19:45:11.813 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-08-25T19:45:11.813 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-08-25T19:45:11.813 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.813 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-25T19:45:11.813 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-08-25T19:45:11.813 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-08-25T19:45:11.813 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.813 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-08-25T19:45:11.813 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-25T19:45:11.813 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-25T19:45:11.813 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.813 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-08-25T19:45:11.814 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-08-25T19:45:11.814 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.814 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-08-25T19:45:11.814 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:11.814 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-08-25T19:45:11.814 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-08-25T19:45:11.814 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-08-25T19:45:11.814 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-08-25T19:45:11.814 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-25T19:45:11.814 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-25T19:45:11.814 INFO:teuthology.orchestra.run.smithi119.stdout: rbd default map options = ms_mode=legacy 2024-08-25T19:45:11.815 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-08-25T19:45:11.815 INFO:teuthology.orchestra.run.smithi121.stdout:[global] 2024-08-25T19:45:11.816 INFO:teuthology.orchestra.run.smithi121.stdout: chdir = "" 2024-08-25T19:45:11.816 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-25T19:45:11.816 INFO:teuthology.orchestra.run.smithi121.stdout: auth supported = cephx 2024-08-25T19:45:11.816 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.816 INFO:teuthology.orchestra.run.smithi121.stdout: filestore xattr use omap = true 2024-08-25T19:45:11.816 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.816 INFO:teuthology.orchestra.run.smithi121.stdout: mon clock drift allowed = 1.000 2024-08-25T19:45:11.816 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.816 INFO:teuthology.orchestra.run.smithi121.stdout: osd crush chooseleaf type = 0 2024-08-25T19:45:11.816 INFO:teuthology.orchestra.run.smithi121.stdout: auth debug = true 2024-08-25T19:45:11.816 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on old message = true 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on bug = true 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn max object skew = 0 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: # disable pg_autoscaler by default for new pools 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default size = 2 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow primary affinity = true 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow pg remap = true 2024-08-25T19:45:11.817 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on legacy crush tunables = false 2024-08-25T19:45:11.818 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on crush straw calc version zero = false 2024-08-25T19:45:11.818 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on no sortbitwise = false 2024-08-25T19:45:11.818 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = false 2024-08-25T19:45:11.818 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on too few osds = false 2024-08-25T19:45:11.818 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-25T19:45:11.818 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-25T19:45:11.818 INFO:teuthology.orchestra.run.smithi121.stdout: mon_allow_pool_size_one = true 2024-08-25T19:45:11.818 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.818 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-25T19:45:11.818 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.818 INFO:teuthology.orchestra.run.smithi121.stdout: osd default data pool replay window = 5 2024-08-25T19:45:11.818 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout: mon allow pool delete = true 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout: mon cluster log file level = debug 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout: debug asserts on shutdown = true 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout: mon health detail to clog = false 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout: mon host = 172.21.15.119 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on pool no app = False 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on skipped message = False 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout: fsid = c5c74996-f4b7-4137-83c8-02707cfa2a6b 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout:[osd] 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout: osd journal size = 100 2024-08-25T19:45:11.819 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub load threshold = 5.0 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub max interval = 600 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock profile = high_recovery_ops 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: osd recover clone overlap = true 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: osd recovery max chunk = 1048576 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug shutdown = true 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug op order = true 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify stray on activate = true 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug trim objects = true 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.820 INFO:teuthology.orchestra.run.smithi121.stdout: osd open classes on start = true 2024-08-25T19:45:11.821 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug pg log writeout = true 2024-08-25T19:45:11.821 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.821 INFO:teuthology.orchestra.run.smithi121.stdout: osd deep scrub update digest min age = 30 2024-08-25T19:45:11.821 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.821 INFO:teuthology.orchestra.run.smithi121.stdout: osd map max advance = 10 2024-08-25T19:45:11.821 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.821 INFO:teuthology.orchestra.run.smithi121.stdout: journal zero on create = true 2024-08-25T19:45:11.821 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.821 INFO:teuthology.orchestra.run.smithi121.stdout: filestore ondisk finisher threads = 3 2024-08-25T19:45:11.821 INFO:teuthology.orchestra.run.smithi121.stdout: filestore apply finisher threads = 3 2024-08-25T19:45:11.821 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.822 INFO:teuthology.orchestra.run.smithi121.stdout: bdev debug aio = true 2024-08-25T19:45:11.822 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug misdirected ops = true 2024-08-25T19:45:11.822 INFO:teuthology.orchestra.run.smithi121.stdout: bdev async discard = True 2024-08-25T19:45:11.822 INFO:teuthology.orchestra.run.smithi121.stdout: bdev enable discard = True 2024-08-25T19:45:11.822 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore allocator = bitmap 2024-08-25T19:45:11.822 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore block size = 96636764160 2024-08-25T19:45:11.822 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore fsck on mount = True 2024-08-25T19:45:11.822 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluefs = 1/20 2024-08-25T19:45:11.822 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluestore = 1/20 2024-08-25T19:45:11.822 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-08-25T19:45:11.822 INFO:teuthology.orchestra.run.smithi121.stdout: debug osd = 20 2024-08-25T19:45:11.822 INFO:teuthology.orchestra.run.smithi121.stdout: debug rocksdb = 4/10 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd backfillfull_ratio = 0.85 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd full ratio = 0.9 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd nearfull ratio = 0.8 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout: osd failsafe full ratio = 0.95 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout: osd objectstore = bluestore 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr] 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout: debug mgr = 20 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-08-25T19:45:11.823 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: mgr/telemetry/nag = false 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout:[mon] 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: debug paxos = 20 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: mon data avail warn = 5 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr mkfs grace = 240 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd reporter subtree level = osd 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd prime pg temp = true 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-08-25T19:45:11.824 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.825 INFO:teuthology.orchestra.run.smithi121.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-25T19:45:11.825 INFO:teuthology.orchestra.run.smithi121.stdout: auth mon ticket ttl = 660 # 11m 2024-08-25T19:45:11.825 INFO:teuthology.orchestra.run.smithi121.stdout: auth service ticket ttl = 240 # 4m 2024-08-25T19:45:11.825 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.825 INFO:teuthology.orchestra.run.smithi121.stdout: # don't complain about insecure global_id in the test suite 2024-08-25T19:45:11.825 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-25T19:45:11.825 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-25T19:45:11.825 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.825 INFO:teuthology.orchestra.run.smithi121.stdout: # 1m isn't quite enough 2024-08-25T19:45:11.825 INFO:teuthology.orchestra.run.smithi121.stdout: mon_down_mkfs_grace = 2m 2024-08-25T19:45:11.825 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.825 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_filestore_osds = false 2024-08-25T19:45:11.826 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:11.826 INFO:teuthology.orchestra.run.smithi121.stdout:[client] 2024-08-25T19:45:11.826 INFO:teuthology.orchestra.run.smithi121.stdout: rgw cache enabled = true 2024-08-25T19:45:11.826 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable ops log = true 2024-08-25T19:45:11.826 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable usage log = true 2024-08-25T19:45:11.826 INFO:teuthology.orchestra.run.smithi121.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-25T19:45:11.826 INFO:teuthology.orchestra.run.smithi121.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-25T19:45:11.826 INFO:teuthology.orchestra.run.smithi121.stdout: rbd default map options = ms_mode=legacy 2024-08-25T19:45:11.826 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.a] 2024-08-25T19:45:11.841 INFO:tasks.ceph:Creating admin key on cluster1.mon.a... 2024-08-25T19:45:11.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster1.keyring 2024-08-25T19:45:11.939 INFO:tasks.ceph:Copying monmap to all nodes... 2024-08-25T19:45:11.939 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:11.939 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/cluster1.keyring of=/dev/stdout 2024-08-25T19:45:11.974 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:11.974 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/cluster1.monmap of=/dev/stdout 2024-08-25T19:45:12.048 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2024-08-25T19:45:12.049 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:12.049 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/cluster1.keyring 2024-08-25T19:45:12.049 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2024-08-25T19:45:12.168 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:12.168 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2024-08-25T19:45:12.244 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi121.front.sepia.ceph.com 2024-08-25T19:45:12.244 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:12.244 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/cluster1.keyring 2024-08-25T19:45:12.244 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2024-08-25T19:45:12.323 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:12.323 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2024-08-25T19:45:12.397 INFO:tasks.ceph:Setting up mon nodes... 2024-08-25T19:45:12.397 INFO:tasks.ceph:Setting up mgr nodes... 2024-08-25T19:45:12.398 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mgr/cluster1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster1-x/keyring 2024-08-25T19:45:12.502 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mgr/cluster1-x/keyring 2024-08-25T19:45:12.510 INFO:tasks.ceph:Setting up mds nodes... 2024-08-25T19:45:12.510 INFO:tasks.ceph_client:Setting up client nodes... 2024-08-25T19:45:12.510 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster1.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.keyring 2024-08-25T19:45:12.605 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/cluster1.client.mirror.keyring 2024-08-25T19:45:12.635 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster1.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.0.keyring 2024-08-25T19:45:12.716 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/cluster1.client.mirror.0.keyring 2024-08-25T19:45:12.744 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-08-25T19:45:12.744 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2024-08-25T19:45:12.744 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/cluster1-0 2024-08-25T19:45:12.800 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2024-08-25T19:45:12.800 INFO:tasks.ceph:role: cluster1.osd.0 2024-08-25T19:45:12.800 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2024-08-25T19:45:12.800 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-25T19:45:12.862 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-25T19:45:12.862 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-25T19:45:12.862 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-25T19:45:12.862 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-25T19:45:12.863 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-25T19:45:12.863 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-08-25T19:45:12.863 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-25T19:45:12.863 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-25T19:45:12.863 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-25T19:45:12.863 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-25T19:45:13.251 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-08-25T19:45:13.255 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-08-25T19:45:13.255 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster1-0 2024-08-25T19:45:13.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-0 2024-08-25T19:45:13.465 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/cluster1-1 2024-08-25T19:45:13.522 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2024-08-25T19:45:13.522 INFO:tasks.ceph:role: cluster1.osd.1 2024-08-25T19:45:13.522 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com 2024-08-25T19:45:13.523 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-25T19:45:13.584 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-25T19:45:13.584 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-25T19:45:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-25T19:45:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-25T19:45:13.585 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-25T19:45:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-08-25T19:45:13.585 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-25T19:45:13.585 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-25T19:45:13.585 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-25T19:45:13.585 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-25T19:45:13.944 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-08-25T19:45:13.948 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-08-25T19:45:13.948 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster1-1 2024-08-25T19:45:14.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-1 2024-08-25T19:45:14.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/cluster1-2 2024-08-25T19:45:14.148 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2024-08-25T19:45:14.148 INFO:tasks.ceph:role: cluster1.osd.2 2024-08-25T19:45:14.148 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com 2024-08-25T19:45:14.148 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-25T19:45:14.210 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-25T19:45:14.210 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-25T19:45:14.210 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-25T19:45:14.210 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-25T19:45:14.210 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-25T19:45:14.210 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-08-25T19:45:14.210 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-25T19:45:14.210 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-25T19:45:14.210 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-25T19:45:14.210 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-25T19:45:14.603 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-08-25T19:45:14.606 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-08-25T19:45:14.607 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster1-2 2024-08-25T19:45:14.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-2 2024-08-25T19:45:14.763 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster1.monmap 2024-08-25T19:45:14.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25T19:45:14.862+0000 7f7154c10640 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-0/keyring: can't open /var/lib/ceph/osd/cluster1-0/keyring: (2) No such file or directory 2024-08-25T19:45:14.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25T19:45:14.868+0000 7f7154c10640 -1 created new key in keyring /var/lib/ceph/osd/cluster1-0/keyring 2024-08-25T19:45:14.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25T19:45:14.868+0000 7f7154c10640 -1 bdev(0x56239860d800 /var/lib/ceph/osd/cluster1-0/block) open stat got: (1) Operation not permitted 2024-08-25T19:45:14.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25T19:45:14.868+0000 7f7154c10640 -1 bluestore(/var/lib/ceph/osd/cluster1-0) _read_fsid unparsable uuid 2024-08-25T19:45:17.878 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-0 2024-08-25T19:45:17.935 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster1.monmap 2024-08-25T19:45:18.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25T19:45:18.025+0000 7f0dfe614640 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-1/keyring: can't open /var/lib/ceph/osd/cluster1-1/keyring: (2) No such file or directory 2024-08-25T19:45:18.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25T19:45:18.026+0000 7f0dfe614640 -1 created new key in keyring /var/lib/ceph/osd/cluster1-1/keyring 2024-08-25T19:45:18.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25T19:45:18.026+0000 7f0dfe614640 -1 bdev(0x55a9c32e1800 /var/lib/ceph/osd/cluster1-1/block) open stat got: (1) Operation not permitted 2024-08-25T19:45:18.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25T19:45:18.026+0000 7f0dfe614640 -1 bluestore(/var/lib/ceph/osd/cluster1-1) _read_fsid unparsable uuid 2024-08-25T19:45:21.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-1 2024-08-25T19:45:21.140 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster1.monmap 2024-08-25T19:45:21.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25T19:45:21.233+0000 7fcd03218640 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-2/keyring: can't open /var/lib/ceph/osd/cluster1-2/keyring: (2) No such file or directory 2024-08-25T19:45:21.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25T19:45:21.233+0000 7fcd03218640 -1 created new key in keyring /var/lib/ceph/osd/cluster1-2/keyring 2024-08-25T19:45:21.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25T19:45:21.234+0000 7fcd03218640 -1 bdev(0x55e3cd1a5800 /var/lib/ceph/osd/cluster1-2/block) open stat got: (1) Operation not permitted 2024-08-25T19:45:21.235 INFO:teuthology.orchestra.run.smithi119.stderr:2024-08-25T19:45:21.234+0000 7fcd03218640 -1 bluestore(/var/lib/ceph/osd/cluster1-2) _read_fsid unparsable uuid 2024-08-25T19:45:24.281 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-2 2024-08-25T19:45:24.338 INFO:tasks.ceph:Reading keys from all nodes... 2024-08-25T19:45:24.338 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:24.338 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mgr/cluster1-x/keyring of=/dev/stdout 2024-08-25T19:45:24.393 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:24.393 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/cluster1-0/keyring of=/dev/stdout 2024-08-25T19:45:24.488 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:24.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/cluster1-1/keyring of=/dev/stdout 2024-08-25T19:45:24.583 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:24.583 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/cluster1-2/keyring of=/dev/stdout 2024-08-25T19:45:24.677 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:24.677 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/cluster1.client.mirror.keyring of=/dev/stdout 2024-08-25T19:45:24.717 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:24.718 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/cluster1.client.mirror.0.keyring of=/dev/stdout 2024-08-25T19:45:24.795 INFO:tasks.ceph:Adding keys to all mons... 2024-08-25T19:45:24.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/cluster1.keyring 2024-08-25T19:45:24.847 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2024-08-25T19:45:24.847 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDIictmEXP0HRAANsfIM3y5egM7hvu4Rt2d8g== 2024-08-25T19:45:24.847 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2024-08-25T19:45:24.847 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDKictmshZvMxAAUwY90GL2IqonStR4RVupTw== 2024-08-25T19:45:24.847 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2024-08-25T19:45:24.847 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDOictmq4CdARAAkF3NfdpmXQx81Qz6TBuyXQ== 2024-08-25T19:45:24.847 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2024-08-25T19:45:24.848 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDRictmDWP8DRAAWN+7PKtSOVIAYcvTp0R6pA== 2024-08-25T19:45:24.848 INFO:teuthology.orchestra.run.smithi119.stdout:[client.mirror] 2024-08-25T19:45:24.848 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDIictm0A0cJBAAQdGBZmkwmI7qGtCWmFVf2w== 2024-08-25T19:45:24.848 INFO:teuthology.orchestra.run.smithi119.stdout:[client.mirror.0] 2024-08-25T19:45:24.848 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDIictmyxe1KhAALoLkROemIZmPx7FdOI0KNA== 2024-08-25T19:45:24.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-25T19:45:24.984 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-25T19:45:25.077 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-25T19:45:25.170 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-25T19:45:25.263 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-25T19:45:25.359 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-25T19:45:25.455 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-08-25T19:45:25.455 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/cluster1-a 2024-08-25T19:45:25.510 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster1 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster1.monmap --keyring /etc/ceph/cluster1.keyring 2024-08-25T19:45:25.619 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster1-a 2024-08-25T19:45:25.674 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/cluster1.monmap 2024-08-25T19:45:25.708 INFO:tasks.ceph:Starting mon daemons in cluster cluster1... 2024-08-25T19:45:25.709 INFO:tasks.ceph.cluster1.mon.a:Restarting daemon 2024-08-25T19:45:25.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster1 -i a 2024-08-25T19:45:25.753 INFO:tasks.ceph.cluster1.mon.a:Started 2024-08-25T19:45:25.753 INFO:tasks.ceph:Starting mgr daemons in cluster cluster1... 2024-08-25T19:45:25.753 INFO:tasks.ceph.cluster1.mgr.x:Restarting daemon 2024-08-25T19:45:25.753 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster1 -i x 2024-08-25T19:45:25.758 INFO:tasks.ceph.cluster1.mgr.x:Started 2024-08-25T19:45:25.759 DEBUG:tasks.ceph:set 0 configs 2024-08-25T19:45:25.759 INFO:tasks.ceph:Setting crush tunables to default 2024-08-25T19:45:25.759 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster cluster1 osd crush tunables default 2024-08-25T19:45:25.987 INFO:teuthology.orchestra.run.smithi119.stderr:adjusted tunables profile to default 2024-08-25T19:45:26.002 INFO:tasks.ceph:check_enable_crimson: False 2024-08-25T19:45:26.002 INFO:tasks.ceph:Starting osd daemons in cluster cluster1... 2024-08-25T19:45:26.002 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:26.002 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/cluster1-0/fsid of=/dev/stdout 2024-08-25T19:45:26.057 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:26.057 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/cluster1-1/fsid of=/dev/stdout 2024-08-25T19:45:26.155 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:26.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/cluster1-2/fsid of=/dev/stdout 2024-08-25T19:45:26.251 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster cluster1 osd new ac2735a0-67f3-4768-a027-9f903902b077 0 2024-08-25T19:45:26.530 INFO:teuthology.orchestra.run.smithi119.stdout:0 2024-08-25T19:45:26.546 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster cluster1 osd new e9fa7c61-90e7-415e-a1cd-f68cb71ac72f 1 2024-08-25T19:45:26.779 INFO:teuthology.orchestra.run.smithi119.stdout:1 2024-08-25T19:45:26.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster cluster1 osd new 234bf606-0a17-42e0-a74a-de1639d9e84d 2 2024-08-25T19:45:27.026 INFO:teuthology.orchestra.run.smithi119.stdout:2 2024-08-25T19:45:27.041 INFO:tasks.ceph.cluster1.osd.0:Restarting daemon 2024-08-25T19:45:27.041 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 0 2024-08-25T19:45:27.045 INFO:tasks.ceph.cluster1.osd.0:Started 2024-08-25T19:45:27.045 INFO:tasks.ceph.cluster1.osd.1:Restarting daemon 2024-08-25T19:45:27.045 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 1 2024-08-25T19:45:27.049 INFO:tasks.ceph.cluster1.osd.1:Started 2024-08-25T19:45:27.049 INFO:tasks.ceph.cluster1.osd.2:Restarting daemon 2024-08-25T19:45:27.049 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 2 2024-08-25T19:45:27.053 INFO:tasks.ceph.cluster1.osd.2:Started 2024-08-25T19:45:27.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2024-08-25T19:45:27.344 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:27.344 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":5,"fsid":"c5c74996-f4b7-4137-83c8-02707cfa2a6b","created":"2024-08-25T19:45:25.903665+0000","modified":"2024-08-25T19:45:27.025216+0000","last_up_change":"0.000000","last_in_change":"2024-08-25T19:45:27.025216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ac2735a0-67f3-4768-a027-9f903902b077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e9fa7c61-90e7-415e-a1cd-f68cb71ac72f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"234bf606-0a17-42e0-a74a-de1639d9e84d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T19:45:27.359 INFO:tasks.ceph.ceph_manager.cluster1:[] 2024-08-25T19:45:27.360 INFO:tasks.ceph:Waiting for OSDs to come up 2024-08-25T19:45:27.661 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2024-08-25T19:45:27.871 INFO:teuthology.misc.health.smithi119.stdout: 2024-08-25T19:45:27.871 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":5,"fsid":"c5c74996-f4b7-4137-83c8-02707cfa2a6b","created":"2024-08-25T19:45:25.903665+0000","modified":"2024-08-25T19:45:27.025216+0000","last_up_change":"0.000000","last_in_change":"2024-08-25T19:45:27.025216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ac2735a0-67f3-4768-a027-9f903902b077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e9fa7c61-90e7-415e-a1cd-f68cb71ac72f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"234bf606-0a17-42e0-a74a-de1639d9e84d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T19:45:27.884 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-08-25T19:45:28.332 INFO:tasks.ceph.cluster1.osd.0.smithi119.stderr:2024-08-25T19:45:28.331+0000 7f3278ee3640 -1 Falling back to public interface 2024-08-25T19:45:28.601 INFO:tasks.ceph.cluster1.osd.1.smithi119.stderr:2024-08-25T19:45:28.600+0000 7fe6a21da640 -1 Falling back to public interface 2024-08-25T19:45:28.617 INFO:tasks.ceph.cluster1.osd.2.smithi119.stderr:2024-08-25T19:45:28.616+0000 7f01364d3640 -1 Falling back to public interface 2024-08-25T19:45:31.979 INFO:tasks.ceph.cluster1.mgr.x.smithi119.stderr:2024-08-25T19:45:31.977+0000 7f4cd7200640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-08-25T19:45:32.275 INFO:tasks.ceph.cluster1.osd.0.smithi119.stderr:2024-08-25T19:45:32.274+0000 7f3278ee3640 -1 osd.0 0 log_to_monitors true 2024-08-25T19:45:32.289 INFO:tasks.ceph.cluster1.osd.1.smithi119.stderr:2024-08-25T19:45:32.287+0000 7fe6a21da640 -1 osd.1 0 log_to_monitors true 2024-08-25T19:45:32.541 INFO:tasks.ceph.cluster1.osd.2.smithi119.stderr:2024-08-25T19:45:32.540+0000 7f01364d3640 -1 osd.2 0 log_to_monitors true 2024-08-25T19:45:34.188 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2024-08-25T19:45:34.605 INFO:teuthology.misc.health.smithi119.stdout: 2024-08-25T19:45:34.605 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":7,"fsid":"c5c74996-f4b7-4137-83c8-02707cfa2a6b","created":"2024-08-25T19:45:25.903665+0000","modified":"2024-08-25T19:45:33.975500+0000","last_up_change":"0.000000","last_in_change":"2024-08-25T19:45:27.025216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ac2735a0-67f3-4768-a027-9f903902b077","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e9fa7c61-90e7-415e-a1cd-f68cb71ac72f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"234bf606-0a17-42e0-a74a-de1639d9e84d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T19:45:34.623 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-08-25T19:45:35.759 INFO:tasks.ceph.cluster1.osd.0.smithi119.stderr:2024-08-25T19:45:35.758+0000 7f3273e00640 -1 osd.0 0 waiting for initial osdmap 2024-08-25T19:45:35.766 INFO:tasks.ceph.cluster1.osd.0.smithi119.stderr:2024-08-25T19:45:35.765+0000 7f3269e00640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-25T19:45:35.809 INFO:tasks.ceph.cluster1.osd.1.smithi119.stderr:2024-08-25T19:45:35.808+0000 7fe69da00640 -1 osd.1 0 waiting for initial osdmap 2024-08-25T19:45:35.816 INFO:tasks.ceph.cluster1.osd.1.smithi119.stderr:2024-08-25T19:45:35.815+0000 7fe693000640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-25T19:45:35.903 INFO:tasks.ceph.cluster1.osd.2.smithi119.stderr:2024-08-25T19:45:35.902+0000 7f0131400640 -1 osd.2 0 waiting for initial osdmap 2024-08-25T19:45:35.918 INFO:tasks.ceph.cluster1.osd.2.smithi119.stderr:2024-08-25T19:45:35.917+0000 7f0127400640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-25T19:45:40.927 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2024-08-25T19:45:41.273 INFO:teuthology.misc.health.smithi119.stdout: 2024-08-25T19:45:41.273 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":11,"fsid":"c5c74996-f4b7-4137-83c8-02707cfa2a6b","created":"2024-08-25T19:45:25.903665+0000","modified":"2024-08-25T19:45:39.992278+0000","last_up_change":"2024-08-25T19:45:36.760528+0000","last_in_change":"2024-08-25T19:45:27.025216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-25T19:45:36.999898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac2735a0-67f3-4768-a027-9f903902b077","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6801","nonce":3716486698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6803","nonce":3716486698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6807","nonce":3716486698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6805","nonce":3716486698}]},"public_addr":"172.21.15.119:6801/3716486698","cluster_addr":"172.21.15.119:6803/3716486698","heartbeat_back_addr":"172.21.15.119:6807/3716486698","heartbeat_front_addr":"172.21.15.119:6805/3716486698","state":["exists","up"]},{"osd":1,"uuid":"e9fa7c61-90e7-415e-a1cd-f68cb71ac72f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6809","nonce":1091288145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6811","nonce":1091288145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6815","nonce":1091288145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6813","nonce":1091288145}]},"public_addr":"172.21.15.119:6809/1091288145","cluster_addr":"172.21.15.119:6811/1091288145","heartbeat_back_addr":"172.21.15.119:6815/1091288145","heartbeat_front_addr":"172.21.15.119:6813/1091288145","state":["exists","up"]},{"osd":2,"uuid":"234bf606-0a17-42e0-a74a-de1639d9e84d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6817","nonce":944614458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6819","nonce":944614458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6823","nonce":944614458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6821","nonce":944614458}]},"public_addr":"172.21.15.119:6817/944614458","cluster_addr":"172.21.15.119:6819/944614458","heartbeat_back_addr":"172.21.15.119:6823/944614458","heartbeat_front_addr":"172.21.15.119:6821/944614458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T19:45:41.287 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-08-25T19:45:41.287 INFO:tasks.ceph:Creating RBD pool 2024-08-25T19:45:41.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster cluster1 osd pool create rbd 8 2024-08-25T19:45:42.005 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'rbd' created 2024-08-25T19:45:42.027 DEBUG:teuthology.orchestra.run.smithi119:> rbd --cluster cluster1 pool init rbd 2024-08-25T19:45:45.035 INFO:tasks.ceph:Starting mds daemons in cluster cluster1... 2024-08-25T19:45:45.036 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 config log 1 --format=json 2024-08-25T19:45:45.036 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-08-25T19:45:45.394 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:45.411 INFO:teuthology.orchestra.run.smithi119.stdout:[{"version":4,"timestamp":"2024-08-25T19:45:35.903954+0000","name":"","changes":[{"name":"osd.2/osd_mclock_max_capacity_iops_ssd","new_value":"4526.870077"}]}] 2024-08-25T19:45:45.412 INFO:tasks.ceph_manager:config epoch is 4 2024-08-25T19:45:45.412 INFO:tasks.ceph:Waiting until cluster1 daemons up and pgs clean... 2024-08-25T19:45:45.412 INFO:tasks.ceph.ceph_manager.cluster1:waiting for mgr available 2024-08-25T19:45:45.412 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 mgr dump --format=json 2024-08-25T19:45:45.807 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:45.824 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1335311570},{"type":"v1","addr":"172.21.15.119:6825","nonce":1335311570}]},"active_addr":"172.21.15.119:6825/1335311570","active_change":"2024-08-25T19:45:30.970066+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3386130859}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1176670171}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":329136925}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3698330659}]}]} 2024-08-25T19:45:45.826 INFO:tasks.ceph.ceph_manager.cluster1:mgr available! 2024-08-25T19:45:45.826 INFO:tasks.ceph.ceph_manager.cluster1:waiting for all up 2024-08-25T19:45:45.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2024-08-25T19:45:46.177 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:46.177 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":15,"fsid":"c5c74996-f4b7-4137-83c8-02707cfa2a6b","created":"2024-08-25T19:45:25.903665+0000","modified":"2024-08-25T19:45:45.018563+0000","last_up_change":"2024-08-25T19:45:36.760528+0000","last_in_change":"2024-08-25T19:45:27.025216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-25T19:45:36.999898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-25T19:45:41.639916+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac2735a0-67f3-4768-a027-9f903902b077","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6801","nonce":3716486698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6803","nonce":3716486698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6807","nonce":3716486698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6805","nonce":3716486698}]},"public_addr":"172.21.15.119:6801/3716486698","cluster_addr":"172.21.15.119:6803/3716486698","heartbeat_back_addr":"172.21.15.119:6807/3716486698","heartbeat_front_addr":"172.21.15.119:6805/3716486698","state":["exists","up"]},{"osd":1,"uuid":"e9fa7c61-90e7-415e-a1cd-f68cb71ac72f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6809","nonce":1091288145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6811","nonce":1091288145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6815","nonce":1091288145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6813","nonce":1091288145}]},"public_addr":"172.21.15.119:6809/1091288145","cluster_addr":"172.21.15.119:6811/1091288145","heartbeat_back_addr":"172.21.15.119:6815/1091288145","heartbeat_front_addr":"172.21.15.119:6813/1091288145","state":["exists","up"]},{"osd":2,"uuid":"234bf606-0a17-42e0-a74a-de1639d9e84d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6817","nonce":944614458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6819","nonce":944614458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6823","nonce":944614458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6821","nonce":944614458}]},"public_addr":"172.21.15.119:6817/944614458","cluster_addr":"172.21.15.119:6819/944614458","heartbeat_back_addr":"172.21.15.119:6823/944614458","heartbeat_front_addr":"172.21.15.119:6821/944614458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:45:33.257695+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:45:33.296871+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:45:33.527084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T19:45:46.195 INFO:tasks.ceph.ceph_manager.cluster1:all up! 2024-08-25T19:45:46.195 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2024-08-25T19:45:46.546 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:46.547 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":15,"fsid":"c5c74996-f4b7-4137-83c8-02707cfa2a6b","created":"2024-08-25T19:45:25.903665+0000","modified":"2024-08-25T19:45:45.018563+0000","last_up_change":"2024-08-25T19:45:36.760528+0000","last_in_change":"2024-08-25T19:45:27.025216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-25T19:45:36.999898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-25T19:45:41.639916+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac2735a0-67f3-4768-a027-9f903902b077","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6801","nonce":3716486698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6803","nonce":3716486698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6807","nonce":3716486698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6805","nonce":3716486698}]},"public_addr":"172.21.15.119:6801/3716486698","cluster_addr":"172.21.15.119:6803/3716486698","heartbeat_back_addr":"172.21.15.119:6807/3716486698","heartbeat_front_addr":"172.21.15.119:6805/3716486698","state":["exists","up"]},{"osd":1,"uuid":"e9fa7c61-90e7-415e-a1cd-f68cb71ac72f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6809","nonce":1091288145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6811","nonce":1091288145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6815","nonce":1091288145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6813","nonce":1091288145}]},"public_addr":"172.21.15.119:6809/1091288145","cluster_addr":"172.21.15.119:6811/1091288145","heartbeat_back_addr":"172.21.15.119:6815/1091288145","heartbeat_front_addr":"172.21.15.119:6813/1091288145","state":["exists","up"]},{"osd":2,"uuid":"234bf606-0a17-42e0-a74a-de1639d9e84d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6817","nonce":944614458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6819","nonce":944614458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6823","nonce":944614458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6821","nonce":944614458}]},"public_addr":"172.21.15.119:6817/944614458","cluster_addr":"172.21.15.119:6819/944614458","heartbeat_back_addr":"172.21.15.119:6823/944614458","heartbeat_front_addr":"172.21.15.119:6821/944614458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:45:33.257695+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:45:33.296871+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:45:33.527084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T19:45:46.564 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.0 flush_pg_stats 2024-08-25T19:45:46.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.1 flush_pg_stats 2024-08-25T19:45:46.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.2 flush_pg_stats 2024-08-25T19:45:46.768 INFO:teuthology.orchestra.run.smithi119.stdout:34359738372 2024-08-25T19:45:46.768 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.1 2024-08-25T19:45:46.773 INFO:teuthology.orchestra.run.smithi119.stdout:34359738373 2024-08-25T19:45:46.773 INFO:teuthology.orchestra.run.smithi119.stdout:34359738372 2024-08-25T19:45:46.773 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2024-08-25T19:45:46.774 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.2 2024-08-25T19:45:47.139 INFO:teuthology.orchestra.run.smithi119.stdout:34359738372 2024-08-25T19:45:47.159 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738372 for osd.1 2024-08-25T19:45:47.159 DEBUG:teuthology.parallel:result is None 2024-08-25T19:45:47.181 INFO:teuthology.orchestra.run.smithi119.stdout:34359738372 2024-08-25T19:45:47.196 INFO:teuthology.orchestra.run.smithi119.stdout:34359738373 2024-08-25T19:45:47.203 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738372 for osd.2 2024-08-25T19:45:47.203 DEBUG:teuthology.parallel:result is None 2024-08-25T19:45:47.214 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738373 got 34359738373 for osd.0 2024-08-25T19:45:47.214 DEBUG:teuthology.parallel:result is None 2024-08-25T19:45:47.214 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2024-08-25T19:45:47.214 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-08-25T19:45:47.600 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:47.600 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-08-25T19:45:47.616 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-08-25T19:45:46.980462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82652,"kb_used_data":1552,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032868,"statfs":{"total":289910292480,"available":289825656832,"internally_reserved":0,"allocated":1589248,"data_stored":1295959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.972329"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.028870+0000","last_change":"2024-08-25T19:45:45.028870+0000","last_active":"2024-08-25T19:45:45.028870+0000","last_peered":"2024-08-25T19:45:45.028870+0000","last_clean":"2024-08-25T19:45:45.028870+0000","last_became_active":"2024-08-25T19:45:43.023302+0000","last_became_peered":"2024-08-25T19:45:43.023302+0000","last_unstale":"2024-08-25T19:45:45.028870+0000","last_undegraded":"2024-08-25T19:45:45.028870+0000","last_fullsized":"2024-08-25T19:45:45.028870+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:01:59.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001138303,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.028517+0000","last_change":"2024-08-25T19:45:45.028517+0000","last_active":"2024-08-25T19:45:45.028517+0000","last_peered":"2024-08-25T19:45:45.028517+0000","last_clean":"2024-08-25T19:45:45.028517+0000","last_became_active":"2024-08-25T19:45:43.022938+0000","last_became_peered":"2024-08-25T19:45:43.022938+0000","last_unstale":"2024-08-25T19:45:45.028517+0000","last_undegraded":"2024-08-25T19:45:45.028517+0000","last_fullsized":"2024-08-25T19:45:45.028517+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:07:13.362353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081442199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.028575+0000","last_change":"2024-08-25T19:45:45.028575+0000","last_active":"2024-08-25T19:45:45.028575+0000","last_peered":"2024-08-25T19:45:45.028575+0000","last_clean":"2024-08-25T19:45:45.028575+0000","last_became_active":"2024-08-25T19:45:43.024909+0000","last_became_peered":"2024-08-25T19:45:43.024909+0000","last_unstale":"2024-08-25T19:45:45.028575+0000","last_undegraded":"2024-08-25T19:45:45.028575+0000","last_fullsized":"2024-08-25T19:45:45.028575+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T04:09:46.949509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089386499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.028879+0000","last_change":"2024-08-25T19:45:45.028879+0000","last_active":"2024-08-25T19:45:45.028879+0000","last_peered":"2024-08-25T19:45:45.028879+0000","last_clean":"2024-08-25T19:45:45.028879+0000","last_became_active":"2024-08-25T19:45:43.022595+0000","last_became_peered":"2024-08-25T19:45:43.022595+0000","last_unstale":"2024-08-25T19:45:45.028879+0000","last_undegraded":"2024-08-25T19:45:45.028879+0000","last_fullsized":"2024-08-25T19:45:45.028879+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:12:22.899460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159848,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"15'2","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.026923+0000","last_change":"2024-08-25T19:45:45.023825+0000","last_active":"2024-08-25T19:45:45.026923+0000","last_peered":"2024-08-25T19:45:45.026923+0000","last_clean":"2024-08-25T19:45:45.026923+0000","last_became_active":"2024-08-25T19:45:43.024429+0000","last_became_peered":"2024-08-25T19:45:43.024429+0000","last_unstale":"2024-08-25T19:45:45.026923+0000","last_undegraded":"2024-08-25T19:45:45.026923+0000","last_fullsized":"2024-08-25T19:45:45.026923+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:32:06.467397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051996799999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.095179+0000","last_change":"2024-08-25T19:45:45.095179+0000","last_active":"2024-08-25T19:45:45.095179+0000","last_peered":"2024-08-25T19:45:45.095179+0000","last_clean":"2024-08-25T19:45:45.095179+0000","last_became_active":"2024-08-25T19:45:43.023906+0000","last_became_peered":"2024-08-25T19:45:43.023906+0000","last_unstale":"2024-08-25T19:45:45.095179+0000","last_undegraded":"2024-08-25T19:45:45.095179+0000","last_fullsized":"2024-08-25T19:45:45.095179+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T02:06:50.439446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041333800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.095342+0000","last_change":"2024-08-25T19:45:45.095342+0000","last_active":"2024-08-25T19:45:45.095342+0000","last_peered":"2024-08-25T19:45:45.095342+0000","last_clean":"2024-08-25T19:45:45.095342+0000","last_became_active":"2024-08-25T19:45:43.023267+0000","last_became_peered":"2024-08-25T19:45:43.023267+0000","last_unstale":"2024-08-25T19:45:45.095342+0000","last_undegraded":"2024-08-25T19:45:45.095342+0000","last_fullsized":"2024-08-25T19:45:45.095342+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:32:06.467397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055963200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"13'1","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.029359+0000","last_change":"2024-08-25T19:45:45.029359+0000","last_active":"2024-08-25T19:45:45.029359+0000","last_peered":"2024-08-25T19:45:45.029359+0000","last_clean":"2024-08-25T19:45:45.029359+0000","last_became_active":"2024-08-25T19:45:43.022303+0000","last_became_peered":"2024-08-25T19:45:43.022303+0000","last_unstale":"2024-08-25T19:45:45.029359+0000","last_undegraded":"2024-08-25T19:45:45.029359+0000","last_fullsized":"2024-08-25T19:45:45.029359+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:45:54.687035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637349,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"10'52","reported_seq":88,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.027529+0000","last_change":"2024-08-25T19:45:39.001678+0000","last_active":"2024-08-25T19:45:45.027529+0000","last_peered":"2024-08-25T19:45:45.027529+0000","last_clean":"2024-08-25T19:45:45.027529+0000","last_became_active":"2024-08-25T19:45:38.999237+0000","last_became_peered":"2024-08-25T19:45:38.999237+0000","last_unstale":"2024-08-25T19:45:45.027529+0000","last_undegraded":"2024-08-25T19:45:45.027529+0000","last_fullsized":"2024-08-25T19:45:45.027529+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:37.987775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:37.987775+0000","last_clean_scrub_stamp":"2024-08-25T19:45:37.987775+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:49:16.221195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":131072,"data_stored":38395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":729088,"data_stored":628782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":729088,"data_stored":628782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T19:45:47.617 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-08-25T19:45:47.960 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:47.961 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-08-25T19:45:47.977 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-08-25T19:45:46.980462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82652,"kb_used_data":1552,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032868,"statfs":{"total":289910292480,"available":289825656832,"internally_reserved":0,"allocated":1589248,"data_stored":1295959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.972329"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.028870+0000","last_change":"2024-08-25T19:45:45.028870+0000","last_active":"2024-08-25T19:45:45.028870+0000","last_peered":"2024-08-25T19:45:45.028870+0000","last_clean":"2024-08-25T19:45:45.028870+0000","last_became_active":"2024-08-25T19:45:43.023302+0000","last_became_peered":"2024-08-25T19:45:43.023302+0000","last_unstale":"2024-08-25T19:45:45.028870+0000","last_undegraded":"2024-08-25T19:45:45.028870+0000","last_fullsized":"2024-08-25T19:45:45.028870+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:01:59.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001138303,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.028517+0000","last_change":"2024-08-25T19:45:45.028517+0000","last_active":"2024-08-25T19:45:45.028517+0000","last_peered":"2024-08-25T19:45:45.028517+0000","last_clean":"2024-08-25T19:45:45.028517+0000","last_became_active":"2024-08-25T19:45:43.022938+0000","last_became_peered":"2024-08-25T19:45:43.022938+0000","last_unstale":"2024-08-25T19:45:45.028517+0000","last_undegraded":"2024-08-25T19:45:45.028517+0000","last_fullsized":"2024-08-25T19:45:45.028517+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:07:13.362353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081442199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.028575+0000","last_change":"2024-08-25T19:45:45.028575+0000","last_active":"2024-08-25T19:45:45.028575+0000","last_peered":"2024-08-25T19:45:45.028575+0000","last_clean":"2024-08-25T19:45:45.028575+0000","last_became_active":"2024-08-25T19:45:43.024909+0000","last_became_peered":"2024-08-25T19:45:43.024909+0000","last_unstale":"2024-08-25T19:45:45.028575+0000","last_undegraded":"2024-08-25T19:45:45.028575+0000","last_fullsized":"2024-08-25T19:45:45.028575+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T04:09:46.949509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089386499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.028879+0000","last_change":"2024-08-25T19:45:45.028879+0000","last_active":"2024-08-25T19:45:45.028879+0000","last_peered":"2024-08-25T19:45:45.028879+0000","last_clean":"2024-08-25T19:45:45.028879+0000","last_became_active":"2024-08-25T19:45:43.022595+0000","last_became_peered":"2024-08-25T19:45:43.022595+0000","last_unstale":"2024-08-25T19:45:45.028879+0000","last_undegraded":"2024-08-25T19:45:45.028879+0000","last_fullsized":"2024-08-25T19:45:45.028879+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:12:22.899460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159848,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"15'2","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.026923+0000","last_change":"2024-08-25T19:45:45.023825+0000","last_active":"2024-08-25T19:45:45.026923+0000","last_peered":"2024-08-25T19:45:45.026923+0000","last_clean":"2024-08-25T19:45:45.026923+0000","last_became_active":"2024-08-25T19:45:43.024429+0000","last_became_peered":"2024-08-25T19:45:43.024429+0000","last_unstale":"2024-08-25T19:45:45.026923+0000","last_undegraded":"2024-08-25T19:45:45.026923+0000","last_fullsized":"2024-08-25T19:45:45.026923+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:32:06.467397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051996799999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.095179+0000","last_change":"2024-08-25T19:45:45.095179+0000","last_active":"2024-08-25T19:45:45.095179+0000","last_peered":"2024-08-25T19:45:45.095179+0000","last_clean":"2024-08-25T19:45:45.095179+0000","last_became_active":"2024-08-25T19:45:43.023906+0000","last_became_peered":"2024-08-25T19:45:43.023906+0000","last_unstale":"2024-08-25T19:45:45.095179+0000","last_undegraded":"2024-08-25T19:45:45.095179+0000","last_fullsized":"2024-08-25T19:45:45.095179+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T02:06:50.439446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041333800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.095342+0000","last_change":"2024-08-25T19:45:45.095342+0000","last_active":"2024-08-25T19:45:45.095342+0000","last_peered":"2024-08-25T19:45:45.095342+0000","last_clean":"2024-08-25T19:45:45.095342+0000","last_became_active":"2024-08-25T19:45:43.023267+0000","last_became_peered":"2024-08-25T19:45:43.023267+0000","last_unstale":"2024-08-25T19:45:45.095342+0000","last_undegraded":"2024-08-25T19:45:45.095342+0000","last_fullsized":"2024-08-25T19:45:45.095342+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:32:06.467397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055963200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"13'1","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.029359+0000","last_change":"2024-08-25T19:45:45.029359+0000","last_active":"2024-08-25T19:45:45.029359+0000","last_peered":"2024-08-25T19:45:45.029359+0000","last_clean":"2024-08-25T19:45:45.029359+0000","last_became_active":"2024-08-25T19:45:43.022303+0000","last_became_peered":"2024-08-25T19:45:43.022303+0000","last_unstale":"2024-08-25T19:45:45.029359+0000","last_undegraded":"2024-08-25T19:45:45.029359+0000","last_fullsized":"2024-08-25T19:45:45.029359+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:45:54.687035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637349,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"10'52","reported_seq":88,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:45:45.027529+0000","last_change":"2024-08-25T19:45:39.001678+0000","last_active":"2024-08-25T19:45:45.027529+0000","last_peered":"2024-08-25T19:45:45.027529+0000","last_clean":"2024-08-25T19:45:45.027529+0000","last_became_active":"2024-08-25T19:45:38.999237+0000","last_became_peered":"2024-08-25T19:45:38.999237+0000","last_unstale":"2024-08-25T19:45:45.027529+0000","last_undegraded":"2024-08-25T19:45:45.027529+0000","last_fullsized":"2024-08-25T19:45:45.027529+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:37.987775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:37.987775+0000","last_clean_scrub_stamp":"2024-08-25T19:45:37.987775+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:49:16.221195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":131072,"data_stored":38395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":729088,"data_stored":628782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":729088,"data_stored":628782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T19:45:47.978 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2024-08-25T19:45:47.978 INFO:tasks.ceph:Waiting until ceph cluster cluster1 is healthy... 2024-08-25T19:45:47.979 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy 2024-08-25T19:45:47.979 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 health --format=json 2024-08-25T19:45:48.364 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:48.364 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-25T19:45:48.381 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy done 2024-08-25T19:45:48.381 INFO:teuthology.run_tasks:Running task ceph... 2024-08-25T19:45:48.391 INFO:tasks.ceph:Creating ceph cluster cluster2... 2024-08-25T19:45:48.391 INFO:tasks.ceph:config {'conf': {'client': {'rbd default map options': 'ms_mode=legacy'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'cluster2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-08-25T19:45:48.392 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544', 'branch': 'main', 'description': 'krbd/mirror/{bluestore-bitmap clusters/2-node conf install/ceph ms_mode$/{legacy} tasks/compare-mirror-images}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7870544', 'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '8618c856c9a644ad80ca70c748bebe0c9b6227d4'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': '2 ceph clusters with 1 mon, 1 mgr and 3 osd each'}], 'name': 'teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'rbd default map options': 'ms_mode=legacy'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12'}, 'extra_system_packages': ['pv']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2'], ['cluster2.mon.a', 'cluster2.mgr.x', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster2.client.mirror', 'cluster2.client.mirror.0']], 'seed': 7857, 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12', 'sleep_before_teardown': 0, 'subset': '0/4', 'suite': 'krbd', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_db4959e44f0252a53b4d8688fbf88b3dda70ea12/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12', 'targets': {'smithi119.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLqR8VvVK77NDOLReMDLA0UunAoSi8mErvCIt51FxQdFsTnMvrtmPsn94raf9Z/ZxUUPgLrklOSIjbZn+LeZtSA=', 'smithi121.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFSv2EVnz87750syph3/YS1Efgvmc3yIYvBNKk3Bi1s9Hk1w8k3ZHfg1lD5TTbZYeIWMXihSQDjR7bzAai1KHj8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '8618c856c9a644ad80ca70c748bebe0c9b6227d4'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'flavor': 'default', 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12', 'extra_system_packages': ['pv']}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'rbd default map options': 'ms_mode=legacy'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12'}}, {'ceph': {'cluster': 'cluster2', 'conf': {'client': {'rbd default map options': 'ms_mode=legacy'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'db4959e44f0252a53b4d8688fbf88b3dda70ea12'}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': False}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': False}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/compare_mirror_images.sh']}, 'env': {'RBD_DEVICE_TYPE': 'krbd', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '3h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '4d4c5e30374204f3b32f867e61bb4e1a3880f672', 'timestamp': '2024-08-24_20:48:16', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-08-25T19:45:48.392 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2024-08-25T19:45:48.396 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2024-08-25T19:45:48.435 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2024-08-25T19:45:48.472 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/run/ceph 2024-08-25T19:45:48.535 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:48.535 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2024-08-25T19:45:48.569 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-25T19:45:48.570 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2024-08-25T19:45:48.644 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-25T19:45:48.645 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-25T19:45:48.645 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-08-25T19:45:48.645 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-25T19:45:48.645 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-08-25T19:45:48.645 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-08-25 19:44:55.236944056 +0000 2024-08-25T19:45:48.645 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-08-25 19:43:04.029439312 +0000 2024-08-25T19:45:48.645 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-08-25 19:43:04.029439312 +0000 2024-08-25T19:45:48.645 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-08-25 19:43:04.029439312 +0000 2024-08-25T19:45:48.645 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-25T19:45:48.736 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-08-25T19:45:48.736 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-08-25T19:45:48.736 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000267635 s, 1.9 MB/s 2024-08-25T19:45:48.739 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-25T19:45:48.775 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2024-08-25T19:45:48.849 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-25T19:45:48.849 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-25T19:45:48.849 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-08-25T19:45:48.849 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-25T19:45:48.849 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-08-25T19:45:48.849 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-08-25 19:44:55.238944029 +0000 2024-08-25T19:45:48.849 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-08-25 19:43:04.531432548 +0000 2024-08-25T19:45:48.849 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-08-25 19:43:04.531432548 +0000 2024-08-25T19:45:48.849 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-08-25 19:43:04.531432548 +0000 2024-08-25T19:45:48.849 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-25T19:45:48.942 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-08-25T19:45:48.942 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-08-25T19:45:48.943 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.00228417 s, 224 kB/s 2024-08-25T19:45:48.946 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-25T19:45:48.982 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2024-08-25T19:45:49.058 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-25T19:45:49.059 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-25T19:45:49.059 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-08-25T19:45:49.059 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-25T19:45:49.059 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-08-25T19:45:49.059 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-08-25 19:44:55.238944029 +0000 2024-08-25T19:45:49.059 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-08-25 19:43:04.993426322 +0000 2024-08-25T19:45:49.059 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-08-25 19:43:04.993426322 +0000 2024-08-25T19:45:49.059 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-08-25 19:43:04.993426322 +0000 2024-08-25T19:45:49.059 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-25T19:45:49.150 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-08-25T19:45:49.150 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-08-25T19:45:49.150 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000278607 s, 1.8 MB/s 2024-08-25T19:45:49.153 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-25T19:45:49.188 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2024-08-25T19:45:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-25T19:45:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-25T19:45:49.264 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-08-25T19:45:49.264 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-25T19:45:49.264 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-08-25T19:45:49.264 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-08-25 19:44:55.239944016 +0000 2024-08-25T19:45:49.264 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-08-25 19:43:05.448420192 +0000 2024-08-25T19:45:49.264 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-08-25 19:43:05.448420192 +0000 2024-08-25T19:45:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-08-25 19:43:05.448420192 +0000 2024-08-25T19:45:49.265 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-25T19:45:49.356 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-08-25T19:45:49.356 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-08-25T19:45:49.356 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000283217 s, 1.8 MB/s 2024-08-25T19:45:49.359 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-25T19:45:49.395 INFO:tasks.ceph:osd dev map: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2024-08-25T19:45:49.395 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2024-08-25T19:45:49.395 INFO:tasks.ceph:Generating config... 2024-08-25T19:45:49.396 INFO:tasks.ceph:[client] rbd default map options = ms_mode=legacy 2024-08-25T19:45:49.396 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-08-25T19:45:49.396 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-08-25T19:45:49.396 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-08-25T19:45:49.396 INFO:tasks.ceph:[mgr] debug ms = 1 2024-08-25T19:45:49.397 INFO:tasks.ceph:[mon] debug mon = 20 2024-08-25T19:45:49.397 INFO:tasks.ceph:[mon] debug ms = 1 2024-08-25T19:45:49.397 INFO:tasks.ceph:[mon] debug paxos = 20 2024-08-25T19:45:49.397 INFO:tasks.ceph:[osd] bdev async discard = True 2024-08-25T19:45:49.397 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-08-25T19:45:49.397 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-08-25T19:45:49.397 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-08-25T19:45:49.397 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-08-25T19:45:49.397 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-08-25T19:45:49.397 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-08-25T19:45:49.397 INFO:tasks.ceph:[osd] debug ms = 1 2024-08-25T19:45:49.397 INFO:tasks.ceph:[osd] debug osd = 20 2024-08-25T19:45:49.397 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-08-25T19:45:49.397 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-08-25T19:45:49.398 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-08-25T19:45:49.398 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-08-25T19:45:49.398 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-08-25T19:45:49.398 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-08-25T19:45:49.398 INFO:tasks.ceph:Setting up cluster2.mon.a... 2024-08-25T19:45:49.398 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster2.keyring 2024-08-25T19:45:49.514 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/cluster2.keyring 2024-08-25T19:45:49.519 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster2.keyring 2024-08-25T19:45:49.604 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2024-08-25T19:45:49.659 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster2.mon.a', '172.21.15.121')] 2024-08-25T19:45:49.659 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.121', 'mon warn on pool no app': False, 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd default map options': 'ms_mode=legacy'}, 'mon.a': {}} 2024-08-25T19:45:49.660 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:49.660 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-08-25T19:45:49.695 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.121 --print /home/ubuntu/cephtest/cluster2.monmap 2024-08-25T19:45:49.804 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster2.monmap 2024-08-25T19:45:49.804 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: generated fsid 2c473bc9-1901-47c2-9407-3117e308ba4c 2024-08-25T19:45:49.805 INFO:teuthology.orchestra.run.smithi121.stdout:setting min_mon_release = quincy 2024-08-25T19:45:49.805 INFO:teuthology.orchestra.run.smithi121.stdout:epoch 0 2024-08-25T19:45:49.805 INFO:teuthology.orchestra.run.smithi121.stdout:fsid 2c473bc9-1901-47c2-9407-3117e308ba4c 2024-08-25T19:45:49.805 INFO:teuthology.orchestra.run.smithi121.stdout:last_changed 2024-08-25T19:45:49.804514+0000 2024-08-25T19:45:49.805 INFO:teuthology.orchestra.run.smithi121.stdout:created 2024-08-25T19:45:49.804514+0000 2024-08-25T19:45:49.805 INFO:teuthology.orchestra.run.smithi121.stdout:min_mon_release 17 (quincy) 2024-08-25T19:45:49.805 INFO:teuthology.orchestra.run.smithi121.stdout:election_strategy: 1 2024-08-25T19:45:49.805 INFO:teuthology.orchestra.run.smithi121.stdout:0: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.a 2024-08-25T19:45:49.805 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster2.monmap (1 monitors) 2024-08-25T19:45:49.807 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-08-25T19:45:49.841 INFO:tasks.ceph:Writing /etc/ceph/cluster2.conf for FSID 2c473bc9-1901-47c2-9407-3117e308ba4c... 2024-08-25T19:45:49.841 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2024-08-25T19:45:49.846 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2024-08-25T19:45:49.946 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-08-25T19:45:49.946 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-08-25T19:45:49.946 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-25T19:45:49.946 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-08-25T19:45:49.946 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.946 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-08-25T19:45:49.946 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.947 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.948 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = 172.21.15.121 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on pool no app = False 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on skipped message = False 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = 2c473bc9-1901-47c2-9407-3117e308ba4c 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-08-25T19:45:49.949 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2024-08-25T19:45:49.950 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-08-25T19:45:49.950 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.950 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-08-25T19:45:49.950 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-08-25T19:45:49.950 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.950 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-08-25T19:45:49.950 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-08-25T19:45:49.950 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-08-25T19:45:49.950 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2024-08-25T19:45:49.950 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.950 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-08-25T19:45:49.950 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: bdev async discard = True 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: bdev enable discard = True 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore allocator = bitmap 2024-08-25T19:45:49.951 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 1/20 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 1/20 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 4/10 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-08-25T19:45:49.952 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-08-25T19:45:49.953 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-08-25T19:45:49.954 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.955 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-08-25T19:45:49.955 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T19:45:49.955 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-08-25T19:45:49.955 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-08-25T19:45:49.955 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-08-25T19:45:49.955 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-08-25T19:45:49.955 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-25T19:45:49.955 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-25T19:45:49.955 INFO:teuthology.orchestra.run.smithi119.stdout: rbd default map options = ms_mode=legacy 2024-08-25T19:45:49.955 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-08-25T19:45:49.977 INFO:teuthology.orchestra.run.smithi121.stdout:[global] 2024-08-25T19:45:49.977 INFO:teuthology.orchestra.run.smithi121.stdout: chdir = "" 2024-08-25T19:45:49.977 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-25T19:45:49.977 INFO:teuthology.orchestra.run.smithi121.stdout: auth supported = cephx 2024-08-25T19:45:49.977 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.977 INFO:teuthology.orchestra.run.smithi121.stdout: filestore xattr use omap = true 2024-08-25T19:45:49.978 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.978 INFO:teuthology.orchestra.run.smithi121.stdout: mon clock drift allowed = 1.000 2024-08-25T19:45:49.978 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.978 INFO:teuthology.orchestra.run.smithi121.stdout: osd crush chooseleaf type = 0 2024-08-25T19:45:49.978 INFO:teuthology.orchestra.run.smithi121.stdout: auth debug = true 2024-08-25T19:45:49.978 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.978 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on old message = true 2024-08-25T19:45:49.978 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on bug = true 2024-08-25T19:45:49.978 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.978 INFO:teuthology.orchestra.run.smithi121.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-25T19:45:49.978 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn max object skew = 0 2024-08-25T19:45:49.978 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: # disable pg_autoscaler by default for new pools 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default size = 2 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow primary affinity = true 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow pg remap = true 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on legacy crush tunables = false 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on crush straw calc version zero = false 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on no sortbitwise = false 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = false 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on too few osds = false 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-25T19:45:49.979 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: mon_allow_pool_size_one = true 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: osd default data pool replay window = 5 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: mon allow pool delete = true 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: mon cluster log file level = debug 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: debug asserts on shutdown = true 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: mon health detail to clog = false 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: mon host = 172.21.15.121 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on pool no app = False 2024-08-25T19:45:49.980 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on skipped message = False 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: fsid = 2c473bc9-1901-47c2-9407-3117e308ba4c 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout:[osd] 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: osd journal size = 100 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub load threshold = 5.0 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub max interval = 600 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock profile = high_recovery_ops 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: osd recover clone overlap = true 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: osd recovery max chunk = 1048576 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug shutdown = true 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug op order = true 2024-08-25T19:45:49.981 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify stray on activate = true 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug trim objects = true 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: osd open classes on start = true 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug pg log writeout = true 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: osd deep scrub update digest min age = 30 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: osd map max advance = 10 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: journal zero on create = true 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: filestore ondisk finisher threads = 3 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: filestore apply finisher threads = 3 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.982 INFO:teuthology.orchestra.run.smithi121.stdout: bdev debug aio = true 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug misdirected ops = true 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: bdev async discard = True 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: bdev enable discard = True 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore allocator = bitmap 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore block size = 96636764160 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore fsck on mount = True 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluefs = 1/20 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluestore = 1/20 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: debug osd = 20 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: debug rocksdb = 4/10 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd backfillfull_ratio = 0.85 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd full ratio = 0.9 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd nearfull ratio = 0.8 2024-08-25T19:45:49.983 INFO:teuthology.orchestra.run.smithi121.stdout: osd failsafe full ratio = 0.95 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout: osd objectstore = bluestore 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr] 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout: debug mgr = 20 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout: mgr/telemetry/nag = false 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout:[mon] 2024-08-25T19:45:49.984 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: debug paxos = 20 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: mon data avail warn = 5 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr mkfs grace = 240 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd reporter subtree level = osd 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd prime pg temp = true 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: auth mon ticket ttl = 660 # 11m 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: auth service ticket ttl = 240 # 4m 2024-08-25T19:45:49.985 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout: # don't complain about insecure global_id in the test suite 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout: # 1m isn't quite enough 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout: mon_down_mkfs_grace = 2m 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_filestore_osds = false 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout:[client] 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout: rgw cache enabled = true 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable ops log = true 2024-08-25T19:45:49.986 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable usage log = true 2024-08-25T19:45:49.987 INFO:teuthology.orchestra.run.smithi121.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-25T19:45:49.987 INFO:teuthology.orchestra.run.smithi121.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-25T19:45:49.987 INFO:teuthology.orchestra.run.smithi121.stdout: rbd default map options = ms_mode=legacy 2024-08-25T19:45:49.987 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.a] 2024-08-25T19:45:50.002 INFO:tasks.ceph:Creating admin key on cluster2.mon.a... 2024-08-25T19:45:50.002 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster2.keyring 2024-08-25T19:45:50.099 INFO:tasks.ceph:Copying monmap to all nodes... 2024-08-25T19:45:50.099 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:50.099 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/cluster2.keyring of=/dev/stdout 2024-08-25T19:45:50.133 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:50.133 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/home/ubuntu/cephtest/cluster2.monmap of=/dev/stdout 2024-08-25T19:45:50.209 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2024-08-25T19:45:50.209 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:50.209 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/cluster2.keyring 2024-08-25T19:45:50.209 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2024-08-25T19:45:50.288 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-08-25T19:45:50.288 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2024-08-25T19:45:50.362 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi121.front.sepia.ceph.com 2024-08-25T19:45:50.362 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:50.363 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/cluster2.keyring 2024-08-25T19:45:50.363 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2024-08-25T19:45:50.439 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:45:50.439 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2024-08-25T19:45:50.514 INFO:tasks.ceph:Setting up mon nodes... 2024-08-25T19:45:50.514 INFO:tasks.ceph:Setting up mgr nodes... 2024-08-25T19:45:50.514 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mgr/cluster2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster2-x/keyring 2024-08-25T19:45:50.655 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mgr/cluster2-x/keyring 2024-08-25T19:45:50.661 INFO:tasks.ceph:Setting up mds nodes... 2024-08-25T19:45:50.662 INFO:tasks.ceph_client:Setting up client nodes... 2024-08-25T19:45:50.662 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster2.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.keyring 2024-08-25T19:45:50.742 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/cluster2.client.mirror.keyring 2024-08-25T19:45:50.769 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster2.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.0.keyring 2024-08-25T19:45:50.849 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/cluster2.client.mirror.0.keyring 2024-08-25T19:45:50.876 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-08-25T19:45:50.877 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2024-08-25T19:45:50.877 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/cluster2-0 2024-08-25T19:45:50.932 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2024-08-25T19:45:50.932 INFO:tasks.ceph:role: cluster2.osd.0 2024-08-25T19:45:50.932 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com 2024-08-25T19:45:50.932 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-25T19:45:50.994 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-25T19:45:50.994 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-25T19:45:50.994 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-25T19:45:50.994 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-25T19:45:50.994 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-25T19:45:50.994 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-08-25T19:45:50.994 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-25T19:45:50.994 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-25T19:45:50.994 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-25T19:45:50.994 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-25T19:45:51.405 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-08-25T19:45:51.410 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-08-25T19:45:51.410 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster2-0 2024-08-25T19:45:51.554 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-0 2024-08-25T19:45:51.618 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/cluster2-1 2024-08-25T19:45:51.673 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2024-08-25T19:45:51.674 INFO:tasks.ceph:role: cluster2.osd.1 2024-08-25T19:45:51.674 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com 2024-08-25T19:45:51.674 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-25T19:45:51.736 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-25T19:45:51.736 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-25T19:45:51.737 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-25T19:45:51.737 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-25T19:45:51.737 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-25T19:45:51.737 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-08-25T19:45:51.737 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-25T19:45:51.737 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-25T19:45:51.737 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-25T19:45:51.737 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-25T19:45:52.098 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-08-25T19:45:52.102 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-08-25T19:45:52.102 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster2-1 2024-08-25T19:45:52.188 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-1 2024-08-25T19:45:52.248 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/cluster2-2 2024-08-25T19:45:52.303 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2024-08-25T19:45:52.303 INFO:tasks.ceph:role: cluster2.osd.2 2024-08-25T19:45:52.303 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com 2024-08-25T19:45:52.303 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-25T19:45:52.365 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-25T19:45:52.365 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-25T19:45:52.365 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-25T19:45:52.365 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-25T19:45:52.365 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-25T19:45:52.365 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-08-25T19:45:52.365 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-25T19:45:52.365 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-25T19:45:52.365 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-25T19:45:52.365 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-25T19:45:52.762 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-08-25T19:45:52.766 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-08-25T19:45:52.766 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster2-2 2024-08-25T19:45:52.866 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-2 2024-08-25T19:45:52.930 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster2.monmap 2024-08-25T19:45:53.038 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25T19:45:53.037+0000 7f9489886640 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-0/keyring: can't open /var/lib/ceph/osd/cluster2-0/keyring: (2) No such file or directory 2024-08-25T19:45:53.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25T19:45:53.042+0000 7f9489886640 -1 created new key in keyring /var/lib/ceph/osd/cluster2-0/keyring 2024-08-25T19:45:53.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25T19:45:53.042+0000 7f9489886640 -1 bdev(0x55b7ced01800 /var/lib/ceph/osd/cluster2-0/block) open stat got: (1) Operation not permitted 2024-08-25T19:45:53.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25T19:45:53.042+0000 7f9489886640 -1 bluestore(/var/lib/ceph/osd/cluster2-0) _read_fsid unparsable uuid 2024-08-25T19:45:56.061 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-0 2024-08-25T19:45:56.117 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster2.monmap 2024-08-25T19:45:56.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25T19:45:56.205+0000 7f57b3e16640 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-1/keyring: can't open /var/lib/ceph/osd/cluster2-1/keyring: (2) No such file or directory 2024-08-25T19:45:56.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25T19:45:56.206+0000 7f57b3e16640 -1 created new key in keyring /var/lib/ceph/osd/cluster2-1/keyring 2024-08-25T19:45:56.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25T19:45:56.206+0000 7f57b3e16640 -1 bdev(0x558881213800 /var/lib/ceph/osd/cluster2-1/block) open stat got: (1) Operation not permitted 2024-08-25T19:45:56.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25T19:45:56.206+0000 7f57b3e16640 -1 bluestore(/var/lib/ceph/osd/cluster2-1) _read_fsid unparsable uuid 2024-08-25T19:45:59.468 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-1 2024-08-25T19:45:59.524 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster2.monmap 2024-08-25T19:45:59.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25T19:45:59.613+0000 7f48bc0b1640 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-2/keyring: can't open /var/lib/ceph/osd/cluster2-2/keyring: (2) No such file or directory 2024-08-25T19:45:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25T19:45:59.613+0000 7f48bc0b1640 -1 created new key in keyring /var/lib/ceph/osd/cluster2-2/keyring 2024-08-25T19:45:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25T19:45:59.613+0000 7f48bc0b1640 -1 bdev(0x564b156c5800 /var/lib/ceph/osd/cluster2-2/block) open stat got: (1) Operation not permitted 2024-08-25T19:45:59.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-08-25T19:45:59.613+0000 7f48bc0b1640 -1 bluestore(/var/lib/ceph/osd/cluster2-2) _read_fsid unparsable uuid 2024-08-25T19:46:02.680 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-2 2024-08-25T19:46:02.737 INFO:tasks.ceph:Reading keys from all nodes... 2024-08-25T19:46:02.737 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:46:02.737 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mgr/cluster2-x/keyring of=/dev/stdout 2024-08-25T19:46:02.793 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:46:02.793 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/cluster2-0/keyring of=/dev/stdout 2024-08-25T19:46:02.887 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:46:02.887 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/cluster2-1/keyring of=/dev/stdout 2024-08-25T19:46:02.982 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:46:02.982 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/cluster2-2/keyring of=/dev/stdout 2024-08-25T19:46:03.077 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:46:03.077 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/cluster2.client.mirror.keyring of=/dev/stdout 2024-08-25T19:46:03.152 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:46:03.152 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/cluster2.client.mirror.0.keyring of=/dev/stdout 2024-08-25T19:46:03.228 INFO:tasks.ceph:Adding keys to all mons... 2024-08-25T19:46:03.228 DEBUG:teuthology.orchestra.run.smithi121:> sudo tee -a /etc/ceph/cluster2.keyring 2024-08-25T19:46:03.319 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.x] 2024-08-25T19:46:03.319 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDuictmk/YWJxAA7G2UQaHNXlPXCWu5CXPzLw== 2024-08-25T19:46:03.319 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.0] 2024-08-25T19:46:03.319 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDxictmG89PAhAApeqdt/C8cKnQSfigOZgyzQ== 2024-08-25T19:46:03.319 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.1] 2024-08-25T19:46:03.320 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQD0ictmlB1TDBAAzU2ULXtp9HZzBrx2FBU4EA== 2024-08-25T19:46:03.320 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.2] 2024-08-25T19:46:03.320 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQD3ictm9lSYJBAAx14lN8Lzm/aPmt2F2zb6oQ== 2024-08-25T19:46:03.320 INFO:teuthology.orchestra.run.smithi121.stdout:[client.mirror] 2024-08-25T19:46:03.320 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDuictmD9RCLBAAOm1fJfiYPRzDfJaJdda//w== 2024-08-25T19:46:03.320 INFO:teuthology.orchestra.run.smithi121.stdout:[client.mirror.0] 2024-08-25T19:46:03.320 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDuictm/genMhAA/AOvUyfwDEYY/5mH57zd7g== 2024-08-25T19:46:03.322 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-25T19:46:03.456 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-25T19:46:03.550 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-25T19:46:03.642 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-25T19:46:03.735 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-25T19:46:03.831 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-25T19:46:03.926 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-08-25T19:46:03.927 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/cluster2-a 2024-08-25T19:46:03.982 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster2 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster2.monmap --keyring /etc/ceph/cluster2.keyring 2024-08-25T19:46:04.090 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster2-a 2024-08-25T19:46:04.145 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/cluster2.monmap 2024-08-25T19:46:04.179 INFO:tasks.ceph:Starting mon daemons in cluster cluster2... 2024-08-25T19:46:04.179 INFO:tasks.ceph.cluster2.mon.a:Restarting daemon 2024-08-25T19:46:04.180 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster2 -i a 2024-08-25T19:46:04.224 INFO:tasks.ceph.cluster2.mon.a:Started 2024-08-25T19:46:04.225 INFO:tasks.ceph:Starting mgr daemons in cluster cluster2... 2024-08-25T19:46:04.225 INFO:tasks.ceph.cluster2.mgr.x:Restarting daemon 2024-08-25T19:46:04.225 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster2 -i x 2024-08-25T19:46:04.228 INFO:tasks.ceph.cluster2.mgr.x:Started 2024-08-25T19:46:04.229 DEBUG:tasks.ceph:set 0 configs 2024-08-25T19:46:04.229 INFO:tasks.ceph:Setting crush tunables to default 2024-08-25T19:46:04.229 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster cluster2 osd crush tunables default 2024-08-25T19:46:04.463 INFO:teuthology.orchestra.run.smithi121.stderr:adjusted tunables profile to default 2024-08-25T19:46:04.478 INFO:tasks.ceph:check_enable_crimson: False 2024-08-25T19:46:04.478 INFO:tasks.ceph:Starting osd daemons in cluster cluster2... 2024-08-25T19:46:04.479 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:46:04.479 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/cluster2-0/fsid of=/dev/stdout 2024-08-25T19:46:04.534 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:46:04.534 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/cluster2-1/fsid of=/dev/stdout 2024-08-25T19:46:04.630 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:46:04.630 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/cluster2-2/fsid of=/dev/stdout 2024-08-25T19:46:04.726 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster cluster2 osd new 0b6d14b3-fc24-424d-978d-f031d2571911 0 2024-08-25T19:46:05.001 INFO:teuthology.orchestra.run.smithi121.stdout:0 2024-08-25T19:46:05.017 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster cluster2 osd new 8139be87-fa16-4528-be86-d917a6662414 1 2024-08-25T19:46:05.262 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-08-25T19:46:05.278 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster cluster2 osd new 3275c148-71c8-4157-8a72-325fa4bcff38 2 2024-08-25T19:46:05.510 INFO:teuthology.orchestra.run.smithi121.stdout:2 2024-08-25T19:46:05.526 INFO:tasks.ceph.cluster2.osd.0:Restarting daemon 2024-08-25T19:46:05.526 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 0 2024-08-25T19:46:05.530 INFO:tasks.ceph.cluster2.osd.0:Started 2024-08-25T19:46:05.530 INFO:tasks.ceph.cluster2.osd.1:Restarting daemon 2024-08-25T19:46:05.530 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 1 2024-08-25T19:46:05.534 INFO:tasks.ceph.cluster2.osd.1:Started 2024-08-25T19:46:05.534 INFO:tasks.ceph.cluster2.osd.2:Restarting daemon 2024-08-25T19:46:05.534 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 2 2024-08-25T19:46:05.538 INFO:tasks.ceph.cluster2.osd.2:Started 2024-08-25T19:46:05.538 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2024-08-25T19:46:05.789 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:46:05.789 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":5,"fsid":"2c473bc9-1901-47c2-9407-3117e308ba4c","created":"2024-08-25T19:46:04.378052+0000","modified":"2024-08-25T19:46:05.510002+0000","last_up_change":"0.000000","last_in_change":"2024-08-25T19:46:05.510002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0b6d14b3-fc24-424d-978d-f031d2571911","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8139be87-fa16-4528-be86-d917a6662414","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3275c148-71c8-4157-8a72-325fa4bcff38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T19:46:05.804 INFO:tasks.ceph.ceph_manager.cluster2:[] 2024-08-25T19:46:05.805 INFO:tasks.ceph:Waiting for OSDs to come up 2024-08-25T19:46:06.107 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2024-08-25T19:46:06.314 INFO:teuthology.misc.health.smithi121.stdout: 2024-08-25T19:46:06.314 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":5,"fsid":"2c473bc9-1901-47c2-9407-3117e308ba4c","created":"2024-08-25T19:46:04.378052+0000","modified":"2024-08-25T19:46:05.510002+0000","last_up_change":"0.000000","last_in_change":"2024-08-25T19:46:05.510002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0b6d14b3-fc24-424d-978d-f031d2571911","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8139be87-fa16-4528-be86-d917a6662414","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3275c148-71c8-4157-8a72-325fa4bcff38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T19:46:06.327 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-08-25T19:46:06.819 INFO:tasks.ceph.cluster2.osd.1.smithi121.stderr:2024-08-25T19:46:06.817+0000 7f5e03da1640 -1 Falling back to public interface 2024-08-25T19:46:06.822 INFO:tasks.ceph.cluster2.osd.0.smithi121.stderr:2024-08-25T19:46:06.820+0000 7fcb6e80d640 -1 Falling back to public interface 2024-08-25T19:46:07.064 INFO:tasks.ceph.cluster2.osd.2.smithi121.stderr:2024-08-25T19:46:07.062+0000 7f8065f2d640 -1 Falling back to public interface 2024-08-25T19:46:10.461 INFO:tasks.ceph.cluster2.mgr.x.smithi121.stderr:2024-08-25T19:46:10.459+0000 7f2801000640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-08-25T19:46:10.788 INFO:tasks.ceph.cluster2.osd.1.smithi121.stderr:2024-08-25T19:46:10.786+0000 7f5e03da1640 -1 osd.1 0 log_to_monitors true 2024-08-25T19:46:11.061 INFO:tasks.ceph.cluster2.osd.0.smithi121.stderr:2024-08-25T19:46:11.060+0000 7fcb6e80d640 -1 osd.0 0 log_to_monitors true 2024-08-25T19:46:11.291 INFO:tasks.ceph.cluster2.osd.2.smithi121.stderr:2024-08-25T19:46:11.289+0000 7f8065f2d640 -1 osd.2 0 log_to_monitors true 2024-08-25T19:46:12.630 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2024-08-25T19:46:13.048 INFO:teuthology.misc.health.smithi121.stdout: 2024-08-25T19:46:13.048 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":7,"fsid":"2c473bc9-1901-47c2-9407-3117e308ba4c","created":"2024-08-25T19:46:04.378052+0000","modified":"2024-08-25T19:46:12.460869+0000","last_up_change":"0.000000","last_in_change":"2024-08-25T19:46:05.510002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0b6d14b3-fc24-424d-978d-f031d2571911","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8139be87-fa16-4528-be86-d917a6662414","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3275c148-71c8-4157-8a72-325fa4bcff38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T19:46:13.063 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-08-25T19:46:14.291 INFO:tasks.ceph.cluster2.osd.1.smithi121.stderr:2024-08-25T19:46:14.289+0000 7f5dfec00640 -1 osd.1 0 waiting for initial osdmap 2024-08-25T19:46:14.297 INFO:tasks.ceph.cluster2.osd.1.smithi121.stderr:2024-08-25T19:46:14.295+0000 7f5df4c00640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-25T19:46:14.347 INFO:tasks.ceph.cluster2.osd.0.smithi121.stderr:2024-08-25T19:46:14.345+0000 7fcb69a00640 -1 osd.0 0 waiting for initial osdmap 2024-08-25T19:46:14.354 INFO:tasks.ceph.cluster2.osd.0.smithi121.stderr:2024-08-25T19:46:14.352+0000 7fcb5f000640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-25T19:46:14.375 INFO:tasks.ceph.cluster2.osd.2.smithi121.stderr:2024-08-25T19:46:14.373+0000 7f8061800640 -1 osd.2 0 waiting for initial osdmap 2024-08-25T19:46:14.381 INFO:tasks.ceph.cluster2.osd.2.smithi121.stderr:2024-08-25T19:46:14.379+0000 7f8056e00640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-25T19:46:19.366 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2024-08-25T19:46:19.707 INFO:teuthology.misc.health.smithi121.stdout: 2024-08-25T19:46:19.707 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":11,"fsid":"2c473bc9-1901-47c2-9407-3117e308ba4c","created":"2024-08-25T19:46:04.378052+0000","modified":"2024-08-25T19:46:18.301163+0000","last_up_change":"2024-08-25T19:46:15.292315+0000","last_in_change":"2024-08-25T19:46:05.510002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-25T19:46:15.482233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b6d14b3-fc24-424d-978d-f031d2571911","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6809","nonce":1885352393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6811","nonce":1885352393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6815","nonce":1885352393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6813","nonce":1885352393}]},"public_addr":"172.21.15.121:6809/1885352393","cluster_addr":"172.21.15.121:6811/1885352393","heartbeat_back_addr":"172.21.15.121:6815/1885352393","heartbeat_front_addr":"172.21.15.121:6813/1885352393","state":["exists","up"]},{"osd":1,"uuid":"8139be87-fa16-4528-be86-d917a6662414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6801","nonce":2034030038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6803","nonce":2034030038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6807","nonce":2034030038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6805","nonce":2034030038}]},"public_addr":"172.21.15.121:6801/2034030038","cluster_addr":"172.21.15.121:6803/2034030038","heartbeat_back_addr":"172.21.15.121:6807/2034030038","heartbeat_front_addr":"172.21.15.121:6805/2034030038","state":["exists","up"]},{"osd":2,"uuid":"3275c148-71c8-4157-8a72-325fa4bcff38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6817","nonce":4005813664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6819","nonce":4005813664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6823","nonce":4005813664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6821","nonce":4005813664}]},"public_addr":"172.21.15.121:6817/4005813664","cluster_addr":"172.21.15.121:6819/4005813664","heartbeat_back_addr":"172.21.15.121:6823/4005813664","heartbeat_front_addr":"172.21.15.121:6821/4005813664","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:46:12.024441+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:46:11.836435+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:46:12.306093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T19:46:19.722 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-08-25T19:46:19.722 INFO:tasks.ceph:Creating RBD pool 2024-08-25T19:46:19.722 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster cluster2 osd pool create rbd 8 2024-08-25T19:46:20.309 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'rbd' created 2024-08-25T19:46:20.330 DEBUG:teuthology.orchestra.run.smithi121:> rbd --cluster cluster2 pool init rbd 2024-08-25T19:46:23.345 INFO:tasks.ceph:Starting mds daemons in cluster cluster2... 2024-08-25T19:46:23.346 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 config log 1 --format=json 2024-08-25T19:46:23.346 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-08-25T19:46:23.702 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:46:23.719 INFO:teuthology.orchestra.run.smithi121.stdout:[{"version":4,"timestamp":"2024-08-25T19:46:14.375731+0000","name":"","changes":[{"name":"osd.2/osd_mclock_max_capacity_iops_ssd","new_value":"4207.019044"}]}] 2024-08-25T19:46:23.720 INFO:tasks.ceph_manager:config epoch is 4 2024-08-25T19:46:23.720 INFO:tasks.ceph:Waiting until cluster2 daemons up and pgs clean... 2024-08-25T19:46:23.720 INFO:tasks.ceph.ceph_manager.cluster2:waiting for mgr available 2024-08-25T19:46:23.720 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 mgr dump --format=json 2024-08-25T19:46:24.115 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:46:24.131 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":4184015723},{"type":"v1","addr":"172.21.15.121:6825","nonce":4184015723}]},"active_addr":"172.21.15.121:6825/4184015723","active_change":"2024-08-25T19:46:09.452249+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":3917983968}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":1580702762}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":2584330358}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":2375706710}]}]} 2024-08-25T19:46:24.133 INFO:tasks.ceph.ceph_manager.cluster2:mgr available! 2024-08-25T19:46:24.134 INFO:tasks.ceph.ceph_manager.cluster2:waiting for all up 2024-08-25T19:46:24.134 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2024-08-25T19:46:24.485 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:46:24.485 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":15,"fsid":"2c473bc9-1901-47c2-9407-3117e308ba4c","created":"2024-08-25T19:46:04.378052+0000","modified":"2024-08-25T19:46:23.315950+0000","last_up_change":"2024-08-25T19:46:15.292315+0000","last_in_change":"2024-08-25T19:46:05.510002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-25T19:46:15.482233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-25T19:46:20.072680+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b6d14b3-fc24-424d-978d-f031d2571911","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6809","nonce":1885352393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6811","nonce":1885352393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6815","nonce":1885352393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6813","nonce":1885352393}]},"public_addr":"172.21.15.121:6809/1885352393","cluster_addr":"172.21.15.121:6811/1885352393","heartbeat_back_addr":"172.21.15.121:6815/1885352393","heartbeat_front_addr":"172.21.15.121:6813/1885352393","state":["exists","up"]},{"osd":1,"uuid":"8139be87-fa16-4528-be86-d917a6662414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6801","nonce":2034030038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6803","nonce":2034030038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6807","nonce":2034030038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6805","nonce":2034030038}]},"public_addr":"172.21.15.121:6801/2034030038","cluster_addr":"172.21.15.121:6803/2034030038","heartbeat_back_addr":"172.21.15.121:6807/2034030038","heartbeat_front_addr":"172.21.15.121:6805/2034030038","state":["exists","up"]},{"osd":2,"uuid":"3275c148-71c8-4157-8a72-325fa4bcff38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6817","nonce":4005813664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6819","nonce":4005813664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6823","nonce":4005813664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6821","nonce":4005813664}]},"public_addr":"172.21.15.121:6817/4005813664","cluster_addr":"172.21.15.121:6819/4005813664","heartbeat_back_addr":"172.21.15.121:6823/4005813664","heartbeat_front_addr":"172.21.15.121:6821/4005813664","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:46:12.024441+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:46:11.836435+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:46:12.306093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T19:46:24.503 INFO:tasks.ceph.ceph_manager.cluster2:all up! 2024-08-25T19:46:24.503 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2024-08-25T19:46:24.853 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:46:24.853 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":15,"fsid":"2c473bc9-1901-47c2-9407-3117e308ba4c","created":"2024-08-25T19:46:04.378052+0000","modified":"2024-08-25T19:46:23.315950+0000","last_up_change":"2024-08-25T19:46:15.292315+0000","last_in_change":"2024-08-25T19:46:05.510002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-25T19:46:15.482233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-25T19:46:20.072680+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b6d14b3-fc24-424d-978d-f031d2571911","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6809","nonce":1885352393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6811","nonce":1885352393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6815","nonce":1885352393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6813","nonce":1885352393}]},"public_addr":"172.21.15.121:6809/1885352393","cluster_addr":"172.21.15.121:6811/1885352393","heartbeat_back_addr":"172.21.15.121:6815/1885352393","heartbeat_front_addr":"172.21.15.121:6813/1885352393","state":["exists","up"]},{"osd":1,"uuid":"8139be87-fa16-4528-be86-d917a6662414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6801","nonce":2034030038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6803","nonce":2034030038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6807","nonce":2034030038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6805","nonce":2034030038}]},"public_addr":"172.21.15.121:6801/2034030038","cluster_addr":"172.21.15.121:6803/2034030038","heartbeat_back_addr":"172.21.15.121:6807/2034030038","heartbeat_front_addr":"172.21.15.121:6805/2034030038","state":["exists","up"]},{"osd":2,"uuid":"3275c148-71c8-4157-8a72-325fa4bcff38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6817","nonce":4005813664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6819","nonce":4005813664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6823","nonce":4005813664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6821","nonce":4005813664}]},"public_addr":"172.21.15.121:6817/4005813664","cluster_addr":"172.21.15.121:6819/4005813664","heartbeat_back_addr":"172.21.15.121:6823/4005813664","heartbeat_front_addr":"172.21.15.121:6821/4005813664","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:46:12.024441+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:46:11.836435+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:46:12.306093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T19:46:24.872 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.0 flush_pg_stats 2024-08-25T19:46:24.873 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.1 flush_pg_stats 2024-08-25T19:46:24.873 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.2 flush_pg_stats 2024-08-25T19:46:25.075 INFO:teuthology.orchestra.run.smithi121.stdout:34359738372 2024-08-25T19:46:25.076 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2024-08-25T19:46:25.093 INFO:teuthology.orchestra.run.smithi121.stdout:34359738373 2024-08-25T19:46:25.093 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2024-08-25T19:46:25.094 INFO:teuthology.orchestra.run.smithi121.stdout:34359738372 2024-08-25T19:46:25.094 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2024-08-25T19:46:25.449 INFO:teuthology.orchestra.run.smithi121.stdout:34359738371 2024-08-25T19:46:25.467 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738372 got 34359738371 for osd.0 2024-08-25T19:46:25.513 INFO:teuthology.orchestra.run.smithi121.stdout:34359738371 2024-08-25T19:46:25.518 INFO:teuthology.orchestra.run.smithi121.stdout:34359738372 2024-08-25T19:46:25.533 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738372 got 34359738371 for osd.2 2024-08-25T19:46:25.537 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738373 got 34359738372 for osd.1 2024-08-25T19:46:26.468 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2024-08-25T19:46:26.534 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2024-08-25T19:46:26.538 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2024-08-25T19:46:26.843 INFO:teuthology.orchestra.run.smithi121.stdout:34359738372 2024-08-25T19:46:26.861 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738372 got 34359738372 for osd.0 2024-08-25T19:46:26.861 DEBUG:teuthology.parallel:result is None 2024-08-25T19:46:26.910 INFO:teuthology.orchestra.run.smithi121.stdout:34359738373 2024-08-25T19:46:26.914 INFO:teuthology.orchestra.run.smithi121.stdout:34359738372 2024-08-25T19:46:26.929 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738373 got 34359738373 for osd.1 2024-08-25T19:46:26.929 DEBUG:teuthology.parallel:result is None 2024-08-25T19:46:26.933 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738372 got 34359738372 for osd.2 2024-08-25T19:46:26.933 DEBUG:teuthology.parallel:result is None 2024-08-25T19:46:26.933 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2024-08-25T19:46:26.933 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-08-25T19:46:27.316 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:46:27.316 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-08-25T19:46:27.333 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-08-25T19:46:25.461881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82656,"kb_used_data":1552,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032864,"statfs":{"total":289910292480,"available":289825652736,"internally_reserved":0,"allocated":1589248,"data_stored":1295959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":82963792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.148748"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.324960+0000","last_change":"2024-08-25T19:46:23.325343+0000","last_active":"2024-08-25T19:46:23.324960+0000","last_peered":"2024-08-25T19:46:23.324960+0000","last_clean":"2024-08-25T19:46:23.324960+0000","last_became_active":"2024-08-25T19:46:21.322148+0000","last_became_peered":"2024-08-25T19:46:21.322148+0000","last_unstale":"2024-08-25T19:46:23.324960+0000","last_undegraded":"2024-08-25T19:46:23.324960+0000","last_fullsized":"2024-08-25T19:46:23.324960+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:02:27.808835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010803760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.325040+0000","last_change":"2024-08-25T19:46:23.325326+0000","last_active":"2024-08-25T19:46:23.325040+0000","last_peered":"2024-08-25T19:46:23.325040+0000","last_clean":"2024-08-25T19:46:23.325040+0000","last_became_active":"2024-08-25T19:46:21.322277+0000","last_became_peered":"2024-08-25T19:46:21.322277+0000","last_unstale":"2024-08-25T19:46:23.325040+0000","last_undegraded":"2024-08-25T19:46:23.325040+0000","last_fullsized":"2024-08-25T19:46:23.325040+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:33:07.769968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098805500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.325394+0000","last_change":"2024-08-25T19:46:23.325616+0000","last_active":"2024-08-25T19:46:23.325394+0000","last_peered":"2024-08-25T19:46:23.325394+0000","last_clean":"2024-08-25T19:46:23.325394+0000","last_became_active":"2024-08-25T19:46:21.322099+0000","last_became_peered":"2024-08-25T19:46:21.322099+0000","last_unstale":"2024-08-25T19:46:23.325394+0000","last_undegraded":"2024-08-25T19:46:23.325394+0000","last_fullsized":"2024-08-25T19:46:23.325394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T03:29:14.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248755,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.324643+0000","last_change":"2024-08-25T19:46:23.324883+0000","last_active":"2024-08-25T19:46:23.324643+0000","last_peered":"2024-08-25T19:46:23.324643+0000","last_clean":"2024-08-25T19:46:23.324643+0000","last_became_active":"2024-08-25T19:46:21.322641+0000","last_became_peered":"2024-08-25T19:46:21.322641+0000","last_unstale":"2024-08-25T19:46:23.324643+0000","last_undegraded":"2024-08-25T19:46:23.324643+0000","last_fullsized":"2024-08-25T19:46:23.324643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T07:29:32.662242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062717600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"15'2","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.319590+0000","last_change":"2024-08-25T19:46:23.319590+0000","last_active":"2024-08-25T19:46:23.319590+0000","last_peered":"2024-08-25T19:46:23.319590+0000","last_clean":"2024-08-25T19:46:23.319590+0000","last_became_active":"2024-08-25T19:46:21.321161+0000","last_became_peered":"2024-08-25T19:46:21.321161+0000","last_unstale":"2024-08-25T19:46:23.319590+0000","last_undegraded":"2024-08-25T19:46:23.319590+0000","last_fullsized":"2024-08-25T19:46:23.319590+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T20:13:55.940616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056347000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.351308+0000","last_change":"2024-08-25T19:46:23.351559+0000","last_active":"2024-08-25T19:46:23.351308+0000","last_peered":"2024-08-25T19:46:23.351308+0000","last_clean":"2024-08-25T19:46:23.351308+0000","last_became_active":"2024-08-25T19:46:21.320531+0000","last_became_peered":"2024-08-25T19:46:21.320531+0000","last_unstale":"2024-08-25T19:46:23.351308+0000","last_undegraded":"2024-08-25T19:46:23.351308+0000","last_fullsized":"2024-08-25T19:46:23.351308+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T20:13:55.940616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046501600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.351366+0000","last_change":"2024-08-25T19:46:23.351749+0000","last_active":"2024-08-25T19:46:23.351366+0000","last_peered":"2024-08-25T19:46:23.351366+0000","last_clean":"2024-08-25T19:46:23.351366+0000","last_became_active":"2024-08-25T19:46:21.319377+0000","last_became_peered":"2024-08-25T19:46:21.319377+0000","last_unstale":"2024-08-25T19:46:23.351366+0000","last_undegraded":"2024-08-25T19:46:23.351366+0000","last_fullsized":"2024-08-25T19:46:23.351366+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:06:47.731173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064495999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"13'1","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.324576+0000","last_change":"2024-08-25T19:46:23.324873+0000","last_active":"2024-08-25T19:46:23.324576+0000","last_peered":"2024-08-25T19:46:23.324576+0000","last_clean":"2024-08-25T19:46:23.324576+0000","last_became_active":"2024-08-25T19:46:21.318722+0000","last_became_peered":"2024-08-25T19:46:21.318722+0000","last_unstale":"2024-08-25T19:46:23.324576+0000","last_undegraded":"2024-08-25T19:46:23.324576+0000","last_fullsized":"2024-08-25T19:46:23.324576+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:29:54.480849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099496299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"10'52","reported_seq":88,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.323771+0000","last_change":"2024-08-25T19:46:17.304647+0000","last_active":"2024-08-25T19:46:23.323771+0000","last_peered":"2024-08-25T19:46:23.323771+0000","last_clean":"2024-08-25T19:46:23.323771+0000","last_became_active":"2024-08-25T19:46:17.304344+0000","last_became_peered":"2024-08-25T19:46:17.304344+0000","last_unstale":"2024-08-25T19:46:23.323771+0000","last_undegraded":"2024-08-25T19:46:23.323771+0000","last_fullsized":"2024-08-25T19:46:23.323771+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:16.293123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:16.293123+0000","last_clean_scrub_stamp":"2024-08-25T19:46:16.293123+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T04:35:50.579834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":131072,"data_stored":38395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":729088,"data_stored":628782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":729088,"data_stored":628782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T19:46:27.333 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-08-25T19:46:27.683 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:46:27.683 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-08-25T19:46:27.699 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-08-25T19:46:27.462206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82656,"kb_used_data":1552,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032864,"statfs":{"total":289910292480,"available":289825652736,"internally_reserved":0,"allocated":1589248,"data_stored":1295959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":82963792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.149073"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.324960+0000","last_change":"2024-08-25T19:46:23.325343+0000","last_active":"2024-08-25T19:46:23.324960+0000","last_peered":"2024-08-25T19:46:23.324960+0000","last_clean":"2024-08-25T19:46:23.324960+0000","last_became_active":"2024-08-25T19:46:21.322148+0000","last_became_peered":"2024-08-25T19:46:21.322148+0000","last_unstale":"2024-08-25T19:46:23.324960+0000","last_undegraded":"2024-08-25T19:46:23.324960+0000","last_fullsized":"2024-08-25T19:46:23.324960+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:02:27.808835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010803760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.325040+0000","last_change":"2024-08-25T19:46:23.325326+0000","last_active":"2024-08-25T19:46:23.325040+0000","last_peered":"2024-08-25T19:46:23.325040+0000","last_clean":"2024-08-25T19:46:23.325040+0000","last_became_active":"2024-08-25T19:46:21.322277+0000","last_became_peered":"2024-08-25T19:46:21.322277+0000","last_unstale":"2024-08-25T19:46:23.325040+0000","last_undegraded":"2024-08-25T19:46:23.325040+0000","last_fullsized":"2024-08-25T19:46:23.325040+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:33:07.769968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098805500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.325394+0000","last_change":"2024-08-25T19:46:23.325616+0000","last_active":"2024-08-25T19:46:23.325394+0000","last_peered":"2024-08-25T19:46:23.325394+0000","last_clean":"2024-08-25T19:46:23.325394+0000","last_became_active":"2024-08-25T19:46:21.322099+0000","last_became_peered":"2024-08-25T19:46:21.322099+0000","last_unstale":"2024-08-25T19:46:23.325394+0000","last_undegraded":"2024-08-25T19:46:23.325394+0000","last_fullsized":"2024-08-25T19:46:23.325394+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T03:29:14.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248755,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.324643+0000","last_change":"2024-08-25T19:46:23.324883+0000","last_active":"2024-08-25T19:46:23.324643+0000","last_peered":"2024-08-25T19:46:23.324643+0000","last_clean":"2024-08-25T19:46:23.324643+0000","last_became_active":"2024-08-25T19:46:21.322641+0000","last_became_peered":"2024-08-25T19:46:21.322641+0000","last_unstale":"2024-08-25T19:46:23.324643+0000","last_undegraded":"2024-08-25T19:46:23.324643+0000","last_fullsized":"2024-08-25T19:46:23.324643+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T07:29:32.662242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062717600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"15'2","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.319590+0000","last_change":"2024-08-25T19:46:23.319590+0000","last_active":"2024-08-25T19:46:23.319590+0000","last_peered":"2024-08-25T19:46:23.319590+0000","last_clean":"2024-08-25T19:46:23.319590+0000","last_became_active":"2024-08-25T19:46:21.321161+0000","last_became_peered":"2024-08-25T19:46:21.321161+0000","last_unstale":"2024-08-25T19:46:23.319590+0000","last_undegraded":"2024-08-25T19:46:23.319590+0000","last_fullsized":"2024-08-25T19:46:23.319590+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T20:13:55.940616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056347000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.351308+0000","last_change":"2024-08-25T19:46:23.351559+0000","last_active":"2024-08-25T19:46:23.351308+0000","last_peered":"2024-08-25T19:46:23.351308+0000","last_clean":"2024-08-25T19:46:23.351308+0000","last_became_active":"2024-08-25T19:46:21.320531+0000","last_became_peered":"2024-08-25T19:46:21.320531+0000","last_unstale":"2024-08-25T19:46:23.351308+0000","last_undegraded":"2024-08-25T19:46:23.351308+0000","last_fullsized":"2024-08-25T19:46:23.351308+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T20:13:55.940616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046501600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.351366+0000","last_change":"2024-08-25T19:46:23.351749+0000","last_active":"2024-08-25T19:46:23.351366+0000","last_peered":"2024-08-25T19:46:23.351366+0000","last_clean":"2024-08-25T19:46:23.351366+0000","last_became_active":"2024-08-25T19:46:21.319377+0000","last_became_peered":"2024-08-25T19:46:21.319377+0000","last_unstale":"2024-08-25T19:46:23.351366+0000","last_undegraded":"2024-08-25T19:46:23.351366+0000","last_fullsized":"2024-08-25T19:46:23.351366+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:06:47.731173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064495999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"13'1","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.324576+0000","last_change":"2024-08-25T19:46:23.324873+0000","last_active":"2024-08-25T19:46:23.324576+0000","last_peered":"2024-08-25T19:46:23.324576+0000","last_clean":"2024-08-25T19:46:23.324576+0000","last_became_active":"2024-08-25T19:46:21.318722+0000","last_became_peered":"2024-08-25T19:46:21.318722+0000","last_unstale":"2024-08-25T19:46:23.324576+0000","last_undegraded":"2024-08-25T19:46:23.324576+0000","last_fullsized":"2024-08-25T19:46:23.324576+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:29:54.480849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099496299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"10'52","reported_seq":88,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-08-25T19:46:23.323771+0000","last_change":"2024-08-25T19:46:17.304647+0000","last_active":"2024-08-25T19:46:23.323771+0000","last_peered":"2024-08-25T19:46:23.323771+0000","last_clean":"2024-08-25T19:46:23.323771+0000","last_became_active":"2024-08-25T19:46:17.304344+0000","last_became_peered":"2024-08-25T19:46:17.304344+0000","last_unstale":"2024-08-25T19:46:23.323771+0000","last_undegraded":"2024-08-25T19:46:23.323771+0000","last_fullsized":"2024-08-25T19:46:23.323771+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:16.293123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:16.293123+0000","last_clean_scrub_stamp":"2024-08-25T19:46:16.293123+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T04:35:50.579834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":131072,"data_stored":38395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":729088,"data_stored":628782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":729088,"data_stored":628782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T19:46:27.700 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2024-08-25T19:46:27.700 INFO:tasks.ceph:Waiting until ceph cluster cluster2 is healthy... 2024-08-25T19:46:27.700 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy 2024-08-25T19:46:27.700 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 health --format=json 2024-08-25T19:46:28.078 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T19:46:28.079 INFO:teuthology.orchestra.run.smithi121.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-25T19:46:28.095 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy done 2024-08-25T19:46:28.096 INFO:teuthology.run_tasks:Running task rbd-mirror... 2024-08-25T19:46:28.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2024-08-25T19:46:28.106 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill rbd-mirror --foreground --cluster cluster1 --id mirror.0 2024-08-25T19:46:28.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2024-08-25T19:46:28.110 INFO:teuthology.run_tasks:Running task rbd-mirror... 2024-08-25T19:46:28.124 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2024-08-25T19:46:28.124 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill rbd-mirror --foreground --cluster cluster2 --id mirror.0 2024-08-25T19:46:28.127 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2024-08-25T19:46:28.128 INFO:teuthology.run_tasks:Running task workunit... 2024-08-25T19:46:28.139 INFO:tasks.workunit:Pulling workunits from ref db4959e44f0252a53b4d8688fbf88b3dda70ea12 2024-08-25T19:46:28.140 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-25T19:46:28.140 DEBUG:teuthology.orchestra.run.smithi121:> stat -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2024-08-25T19:46:28.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-25T19:46:28.178 INFO:teuthology.orchestra.run.smithi121.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.cluster1.mirror': No such file or directory 2024-08-25T19:46:28.179 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2024-08-25T19:46:28.256 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.cluster1.mirror 2024-08-25T19:46:28.256 DEBUG:teuthology.orchestra.run.smithi121:> cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror && mkdir -- client.mirror 2024-08-25T19:46:28.331 INFO:tasks.workunit:timeout=3h 2024-08-25T19:46:28.331 INFO:tasks.workunit:cleanup=True 2024-08-25T19:46:28.332 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf /home/ubuntu/cephtest/clone.cluster1.client.mirror && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.cluster1.client.mirror && cd /home/ubuntu/cephtest/clone.cluster1.client.mirror && git checkout db4959e44f0252a53b4d8688fbf88b3dda70ea12 2024-08-25T19:46:28.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:Cloning into '/home/ubuntu/cephtest/clone.cluster1.client.mirror'... 2024-08-25T19:47:30.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:Updating files: 65% (8432/12879) Updating files: 66% (8501/12879) Updating files: 67% (8629/12879) Updating files: 68% (8758/12879) Updating files: 69% (8887/12879) Updating files: 70% (9016/12879) Updating files: 71% (9145/12879) Updating files: 72% (9273/12879) Updating files: 73% (9402/12879) Updating files: 74% (9531/12879) Updating files: 75% (9660/12879) Updating files: 76% (9789/12879) Updating files: 77% (9917/12879) Updating files: 78% (10046/12879) Updating files: 79% (10175/12879) Updating files: 80% (10304/12879) Updating files: 81% (10432/12879) Updating files: 82% (10561/12879) Updating files: 83% (10690/12879) Updating files: 84% (10819/12879) Updating files: 85% (10948/12879) Updating files: 86% (11076/12879) Updating files: 87% (11205/12879) Updating files: 88% (11334/12879) Updating files: 89% (11463/12879) Updating files: 90% (11592/12879) Updating files: 91% (11720/12879) Updating files: 92% (11849/12879) Updating files: 93% (11978/12879) Updating files: 94% (12107/12879) Updating files: 95% (12236/12879) Updating files: 96% (12364/12879) Updating files: 97% (12493/12879) Updating files: 98% (12622/12879) Updating files: 99% (12751/12879) Updating files: 100% (12879/12879) Updating files: 100% (12879/12879), done. 2024-08-25T19:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:Note: switching to 'db4959e44f0252a53b4d8688fbf88b3dda70ea12'. 2024-08-25T19:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2024-08-25T19:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-25T19:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-25T19:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:state without impacting any branches by switching back to a branch. 2024-08-25T19:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2024-08-25T19:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-25T19:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-25T19:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2024-08-25T19:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: git switch -c 2024-08-25T19:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2024-08-25T19:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:Or undo this operation with: 2024-08-25T19:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2024-08-25T19:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: git switch - 2024-08-25T19:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2024-08-25T19:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-25T19:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2024-08-25T19:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:HEAD is now at db4959e44f0 Merge PR #58487 into main 2024-08-25T19:47:30.616 DEBUG:teuthology.orchestra.run.smithi121:> cd -- /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror 2024-08-25T19:47:30.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-25T19:47:30.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2024-08-25T19:47:30.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-25T19:47:31.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-25T19:47:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-25T19:47:31.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2024-08-25T19:47:31.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2024-08-25T19:47:31.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-25T19:47:31.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2024-08-25T19:47:31.578 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-08-25T19:47:31.578 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror of=/dev/stdout 2024-08-25T19:47:31.614 INFO:tasks.workunit:Running workunits matching rbd/compare_mirror_images.sh on cluster1.client.mirror... 2024-08-25T19:47:31.616 INFO:tasks.workunit:Running workunit rbd/compare_mirror_images.sh... 2024-08-25T19:47:31.616 DEBUG:teuthology.orchestra.run.smithi121:workunit test rbd/compare_mirror_images.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=db4959e44f0252a53b4d8688fbf88b3dda70ea12 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror RBD_DEVICE_TYPE=krbd RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/compare_mirror_images.sh 2024-08-25T19:47:31.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG_PREFIX=image-primary 2024-08-25T19:47:31.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MIRROR_IMAGE_MODE=snapshot 2024-08-25T19:47:31.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MIRROR_POOL_MODE=image 2024-08-25T19:47:31.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT_PREFIX=test-primary 2024-08-25T19:47:31.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff 2024-08-25T19:47:31.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RBD_MIRROR_INSTANCES=1 2024-08-25T19:47:31.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RBD_MIRROR_MODE=snapshot 2024-08-25T19:47:31.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RBD_MIRROR_USE_EXISTING_CLUSTER=1 2024-08-25T19:47:31.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/compare_mirror_images.sh 2024-08-25T19:47:31.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ . /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_helpers.sh 2024-08-25T19:47:31.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ type xmlstarlet 2024-08-25T19:47:31.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ XMLSTARLET=xmlstarlet 2024-08-25T19:47:31.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ RBD_MIRROR_INSTANCES=1 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CLUSTER1=cluster1 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CLUSTER2=cluster2 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ PEER_CLUSTER_SUFFIX= 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ POOL=mirror 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ PARENT_POOL=mirror_parent 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ NS1=ns1 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ NS2=ns2 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ TEMPDIR= 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ID=mirror 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ MIRROR_USER_ID_PREFIX=mirror. 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ RBD_MIRROR_MODE=snapshot 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ MIRROR_POOL_MODE=image 2024-08-25T19:47:31.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ '[' snapshot = snapshot ']' 2024-08-25T19:47:31.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ MIRROR_POOL_MODE=image 2024-08-25T19:47:31.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ export 'CEPH_ARGS=--id mirror' 2024-08-25T19:47:31.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS='--id mirror' 2024-08-25T19:47:31.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ LAST_MIRROR_INSTANCE=0 2024-08-25T19:47:31.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++++ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/compare_mirror_images.sh 2024-08-25T19:47:31.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+++ readlink -f /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/../../../src 2024-08-25T19:47:31.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror/src 2024-08-25T19:47:31.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_BIN=. 2024-08-25T19:47:31.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_SRC=. 2024-08-25T19:47:31.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ '[' -e CMakeCache.txt ']' 2024-08-25T19:47:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ '[' 0 -gt 0 ']' 2024-08-25T19:47:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ setup 2024-08-25T19:47:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local c 2024-08-25T19:47:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ trap 'cleanup $?' INT TERM EXIT 2024-08-25T19:47:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ setup_tempdir 2024-08-25T19:47:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:47:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ mktemp -d 2024-08-25T19:47:31.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ TEMPDIR=/tmp/tmp.vVBlul05Au 2024-08-25T19:47:31.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -z 1 ']' 2024-08-25T19:47:31.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ setup_pools cluster1 cluster2 2024-08-25T19:47:31.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:31.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local remote_cluster=cluster2 2024-08-25T19:47:31.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mon_map_file 2024-08-25T19:47:31.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mon_addr 2024-08-25T19:47:31.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local admin_key_file 2024-08-25T19:47:31.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local uuid 2024-08-25T19:47:31.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T19:47:31.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ceph --cluster cluster1 osd pool create mirror 64 64 2024-08-25T19:47:32.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:pool 'mirror' already exists 2024-08-25T19:47:32.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T19:47:32.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ceph --cluster cluster1 osd pool create mirror_parent 64 64 2024-08-25T19:47:33.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:pool 'mirror_parent' already exists 2024-08-25T19:47:33.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T19:47:33.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 pool init mirror 2024-08-25T19:47:36.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T19:47:36.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 pool init mirror_parent 2024-08-25T19:47:39.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:47:39.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T19:47:39.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror pool enable --site-name cluster1 mirror image 2024-08-25T19:47:39.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror pool enable mirror_parent image 2024-08-25T19:47:39.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 namespace create mirror/ns1 2024-08-25T19:47:39.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 namespace create mirror/ns2 2024-08-25T19:47:39.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns1 image 2024-08-25T19:47:39.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns2 image 2024-08-25T19:47:39.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -z ']' 2024-08-25T19:47:39.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -z ']' 2024-08-25T19:47:39.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ peer_add cluster1 mirror cluster2 2024-08-25T19:47:39.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:39.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:39.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:39.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:39.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local client_cluster=cluster2 2024-08-25T19:47:39.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:39.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local remote_cluster=cluster2 2024-08-25T19:47:39.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local uuid_var_name 2024-08-25T19:47:39.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:47:39.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local error_code 2024-08-25T19:47:39.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local peer_uuid 2024-08-25T19:47:39.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 16 32 2024-08-25T19:47:39.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ set +e 2024-08-25T19:47:39.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool peer add mirror cluster2 2024-08-25T19:47:39.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ peer_uuid=47069a6b-ffb4-4e8d-97c6-277f74945b45 2024-08-25T19:47:39.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ error_code=0 2024-08-25T19:47:39.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ set -e 2024-08-25T19:47:39.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' 0 -eq 17 ']' 2024-08-25T19:47:39.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test 0 -eq 0 2024-08-25T19:47:39.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:47:39.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:47:39.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ peer_add cluster1 mirror_parent cluster2 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror_parent 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local client_cluster=cluster2 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local remote_cluster=cluster2 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local uuid_var_name 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local error_code 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local peer_uuid 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 16 32 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ set +e 2024-08-25T19:47:39.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool peer add mirror_parent cluster2 2024-08-25T19:47:39.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ peer_uuid=ac960e58-439a-4d0d-b907-9f835c3f50d0 2024-08-25T19:47:39.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ error_code=0 2024-08-25T19:47:39.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ set -e 2024-08-25T19:47:39.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' 0 -eq 17 ']' 2024-08-25T19:47:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test 0 -eq 0 2024-08-25T19:47:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:47:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:47:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ setup_pools cluster2 cluster1 2024-08-25T19:47:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:47:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local remote_cluster=cluster1 2024-08-25T19:47:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mon_map_file 2024-08-25T19:47:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mon_addr 2024-08-25T19:47:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local admin_key_file 2024-08-25T19:47:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local uuid 2024-08-25T19:47:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T19:47:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ceph --cluster cluster2 osd pool create mirror 64 64 2024-08-25T19:47:40.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:pool 'mirror' already exists 2024-08-25T19:47:40.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T19:47:40.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ceph --cluster cluster2 osd pool create mirror_parent 64 64 2024-08-25T19:47:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:pool 'mirror_parent' already exists 2024-08-25T19:47:41.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T19:47:41.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster2 pool init mirror 2024-08-25T19:47:44.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T19:47:44.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster2 pool init mirror_parent 2024-08-25T19:47:47.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:47:47.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T19:47:47.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster2 mirror pool enable --site-name cluster2 mirror image 2024-08-25T19:47:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster2 mirror pool enable mirror_parent image 2024-08-25T19:47:47.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster2 namespace create mirror/ns1 2024-08-25T19:47:47.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster2 namespace create mirror/ns2 2024-08-25T19:47:47.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns1 image 2024-08-25T19:47:47.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns2 image 2024-08-25T19:47:47.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -z ']' 2024-08-25T19:47:47.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -z ']' 2024-08-25T19:47:47.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ peer_add cluster2 mirror cluster1 2024-08-25T19:47:47.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:47:47.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local client_cluster=cluster1 2024-08-25T19:47:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local remote_cluster=cluster1 2024-08-25T19:47:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local uuid_var_name 2024-08-25T19:47:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:47:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local error_code 2024-08-25T19:47:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local peer_uuid 2024-08-25T19:47:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 16 32 2024-08-25T19:47:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ set +e 2024-08-25T19:47:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool peer add mirror cluster1 2024-08-25T19:47:48.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ peer_uuid=443f61bb-0997-4627-8ee2-b64044be9781 2024-08-25T19:47:48.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ error_code=0 2024-08-25T19:47:48.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ set -e 2024-08-25T19:47:48.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' 0 -eq 17 ']' 2024-08-25T19:47:48.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test 0 -eq 0 2024-08-25T19:47:48.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ peer_add cluster2 mirror_parent cluster1 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror_parent 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local client_cluster=cluster1 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local remote_cluster=cluster1 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local uuid_var_name 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local error_code 2024-08-25T19:47:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local peer_uuid 2024-08-25T19:47:48.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 16 32 2024-08-25T19:47:48.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ set +e 2024-08-25T19:47:48.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool peer add mirror_parent cluster1 2024-08-25T19:47:48.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ peer_uuid=b74732e8-3233-4350-bc57-b8eec0da09de 2024-08-25T19:47:48.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ error_code=0 2024-08-25T19:47:48.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ set -e 2024-08-25T19:47:48.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' 0 -eq 17 ']' 2024-08-25T19:47:48.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test 0 -eq 0 2024-08-25T19:47:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:47:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:47:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:47:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ start_mirrors cluster1 2024-08-25T19:47:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ seq 0 0 2024-08-25T19:47:48.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-08-25T19:47:48.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ start_mirror cluster1:0 2024-08-25T19:47:48.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1:0 2024-08-25T19:47:48.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local instance 2024-08-25T19:47:48.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-08-25T19:47:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local val=cluster1:0 2024-08-25T19:47:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster_var_name=cluster 2024-08-25T19:47:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local instance_var_name=instance 2024-08-25T19:47:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cluster=cluster1 2024-08-25T19:47:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ instance=0 2024-08-25T19:47:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' 0 = cluster1:0 ']' 2024-08-25T19:47:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ eval cluster=cluster1 2024-08-25T19:47:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ cluster=cluster1 2024-08-25T19:47:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ eval instance=0 2024-08-25T19:47:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ instance=0 2024-08-25T19:47:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test -n 1 2024-08-25T19:47:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 2024-08-25T19:47:48.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ start_mirrors cluster2 2024-08-25T19:47:48.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:47:48.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ seq 0 0 2024-08-25T19:47:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-08-25T19:47:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ start_mirror cluster2:0 2024-08-25T19:47:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2:0 2024-08-25T19:47:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local instance 2024-08-25T19:47:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ set_cluster_instance cluster2:0 cluster instance 2024-08-25T19:47:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local val=cluster2:0 2024-08-25T19:47:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster_var_name=cluster 2024-08-25T19:47:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local instance_var_name=instance 2024-08-25T19:47:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cluster=cluster2 2024-08-25T19:47:48.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ instance=0 2024-08-25T19:47:48.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' 0 = cluster2:0 ']' 2024-08-25T19:47:48.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ eval cluster=cluster2 2024-08-25T19:47:48.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ cluster=cluster2 2024-08-25T19:47:48.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ eval instance=0 2024-08-25T19:47:48.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ instance=0 2024-08-25T19:47:48.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test -n 1 2024-08-25T19:47:48.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 2024-08-25T19:47:48.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wget https://download.ceph.com/qa/linux-5.4.tar.gz 2024-08-25T19:47:48.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:--2024-08-25 19:47:48-- https://download.ceph.com/qa/linux-5.4.tar.gz 2024-08-25T19:47:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-08-25T19:47:48.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:443... connected. 2024-08-25T19:47:48.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:HTTP request sent, awaiting response... 200 OK 2024-08-25T19:47:48.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:Length: 172616875 (165M) [application/octet-stream] 2024-08-25T19:47:48.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:Saving to: ‘linux-5.4.tar.gz’ 2024-08-25T19:47:48.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2024-08-25T19:47:48.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 0K .......... .......... .......... .......... .......... 0% 2.39M 69s 2024-08-25T19:47:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50K .......... .......... .......... .......... .......... 0% 2.38M 69s 2024-08-25T19:47:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 100K .......... .......... .......... .......... .......... 0% 182M 46s 2024-08-25T19:47:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 150K .......... .......... .......... .......... .......... 0% 2.42M 52s 2024-08-25T19:47:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 200K .......... .......... .......... .......... .......... 0% 150M 42s 2024-08-25T19:47:48.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 250K .......... .......... .......... .......... .......... 0% 21.6M 36s 2024-08-25T19:47:48.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 300K .......... .......... .......... .......... .......... 0% 187M 31s 2024-08-25T19:47:48.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 350K .......... .......... .......... .......... .......... 0% 2.74M 34s 2024-08-25T19:47:48.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 400K .......... .......... .......... .......... .......... 0% 185M 31s 2024-08-25T19:47:48.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 450K .......... .......... .......... .......... .......... 0% 184M 28s 2024-08-25T19:47:48.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 500K .......... .......... .......... .......... .......... 0% 214M 25s 2024-08-25T19:47:48.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 550K .......... .......... .......... .......... .......... 0% 161M 23s 2024-08-25T19:47:48.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 600K .......... .......... .......... .......... .......... 0% 197M 22s 2024-08-25T19:47:48.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 650K .......... .......... .......... .......... .......... 0% 196M 20s 2024-08-25T19:47:48.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 700K .......... .......... .......... .......... .......... 0% 95.0M 19s 2024-08-25T19:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 750K .......... .......... .......... .......... .......... 0% 2.71M 21s 2024-08-25T19:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 800K .......... .......... .......... .......... .......... 0% 173M 20s 2024-08-25T19:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 850K .......... .......... .......... .......... .......... 0% 194M 19s 2024-08-25T19:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 900K .......... .......... .......... .......... .......... 0% 217M 18s 2024-08-25T19:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 950K .......... .......... .......... .......... .......... 0% 117M 17s 2024-08-25T19:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1000K .......... .......... .......... .......... .......... 0% 195M 17s 2024-08-25T19:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1050K .......... .......... .......... .......... .......... 0% 210M 16s 2024-08-25T19:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1100K .......... .......... .......... .......... .......... 0% 30.0M 15s 2024-08-25T19:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1150K .......... .......... .......... .......... .......... 0% 20.9M 15s 2024-08-25T19:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1200K .......... .......... .......... .......... .......... 0% 27.4M 15s 2024-08-25T19:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1250K .......... .......... .......... .......... .......... 0% 162M 14s 2024-08-25T19:47:48.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1300K .......... .......... .......... .......... .......... 0% 24.2M 14s 2024-08-25T19:47:48.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1350K .......... .......... .......... .......... .......... 0% 36.4M 14s 2024-08-25T19:47:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1400K .......... .......... .......... .......... .......... 0% 23.5M 13s 2024-08-25T19:47:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1450K .......... .......... .......... .......... .......... 0% 70.4M 13s 2024-08-25T19:47:48.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1500K .......... .......... .......... .......... .......... 0% 26.0M 13s 2024-08-25T19:47:48.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1550K .......... .......... .......... .......... .......... 0% 9.81M 13s 2024-08-25T19:47:48.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1600K .......... .......... .......... .......... .......... 0% 189M 12s 2024-08-25T19:47:48.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1650K .......... .......... .......... .......... .......... 1% 37.3M 12s 2024-08-25T19:47:48.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1700K .......... .......... .......... .......... .......... 1% 24.3M 12s 2024-08-25T19:47:48.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1750K .......... .......... .......... .......... .......... 1% 130M 12s 2024-08-25T19:47:48.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1800K .......... .......... .......... .......... .......... 1% 23.2M 12s 2024-08-25T19:47:48.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1850K .......... .......... .......... .......... .......... 1% 23.8M 11s 2024-08-25T19:47:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1900K .......... .......... .......... .......... .......... 1% 20.8M 11s 2024-08-25T19:47:48.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 1950K .......... .......... .......... .......... .......... 1% 27.0M 11s 2024-08-25T19:47:48.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2000K .......... .......... .......... .......... .......... 1% 170M 11s 2024-08-25T19:47:48.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2050K .......... .......... .......... .......... .......... 1% 24.9M 11s 2024-08-25T19:47:48.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2100K .......... .......... .......... .......... .......... 1% 20.8M 11s 2024-08-25T19:47:48.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2150K .......... .......... .......... .......... .......... 1% 23.8M 11s 2024-08-25T19:47:48.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2200K .......... .......... .......... .......... .......... 1% 56.0M 11s 2024-08-25T19:47:48.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2250K .......... .......... .......... .......... .......... 1% 43.1M 10s 2024-08-25T19:47:48.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2300K .......... .......... .......... .......... .......... 1% 20.4M 10s 2024-08-25T19:47:48.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2350K .......... .......... .......... .......... .......... 1% 13.7M 10s 2024-08-25T19:47:48.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2400K .......... .......... .......... .......... .......... 1% 24.6M 10s 2024-08-25T19:47:48.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2450K .......... .......... .......... .......... .......... 1% 14.5M 10s 2024-08-25T19:47:48.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2500K .......... .......... .......... .......... .......... 1% 110M 10s 2024-08-25T19:47:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2550K .......... .......... .......... .......... .......... 1% 28.1M 10s 2024-08-25T19:47:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2600K .......... .......... .......... .......... .......... 1% 22.9M 10s 2024-08-25T19:47:48.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2650K .......... .......... .......... .......... .......... 1% 23.6M 10s 2024-08-25T19:47:48.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2700K .......... .......... .......... .......... .......... 1% 193M 10s 2024-08-25T19:47:48.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2750K .......... .......... .......... .......... .......... 1% 11.5M 10s 2024-08-25T19:47:48.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2800K .......... .......... .......... .......... .......... 1% 170M 10s 2024-08-25T19:47:48.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2850K .......... .......... .......... .......... .......... 1% 25.1M 10s 2024-08-25T19:47:48.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2900K .......... .......... .......... .......... .......... 1% 22.7M 10s 2024-08-25T19:47:48.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2950K .......... .......... .......... .......... .......... 1% 22.4M 10s 2024-08-25T19:47:48.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3000K .......... .......... .......... .......... .......... 1% 195M 9s 2024-08-25T19:47:48.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3050K .......... .......... .......... .......... .......... 1% 21.4M 9s 2024-08-25T19:47:48.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3100K .......... .......... .......... .......... .......... 1% 24.9M 9s 2024-08-25T19:47:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3150K .......... .......... .......... .......... .......... 1% 16.3M 9s 2024-08-25T19:47:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3200K .......... .......... .......... .......... .......... 1% 30.6M 9s 2024-08-25T19:47:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3250K .......... .......... .......... .......... .......... 1% 178M 9s 2024-08-25T19:47:48.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3300K .......... .......... .......... .......... .......... 1% 15.3M 9s 2024-08-25T19:47:48.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3350K .......... .......... .......... .......... .......... 2% 19.4M 9s 2024-08-25T19:47:48.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3400K .......... .......... .......... .......... .......... 2% 19.9M 9s 2024-08-25T19:47:48.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3450K .......... .......... .......... .......... .......... 2% 29.0M 9s 2024-08-25T19:47:48.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3500K .......... .......... .......... .......... .......... 2% 181M 9s 2024-08-25T19:47:48.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3550K .......... .......... .......... .......... .......... 2% 25.0M 9s 2024-08-25T19:47:48.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3600K .......... .......... .......... .......... .......... 2% 23.7M 9s 2024-08-25T19:47:48.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3650K .......... .......... .......... .......... .......... 2% 23.6M 9s 2024-08-25T19:47:48.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3700K .......... .......... .......... .......... .......... 2% 30.8M 9s 2024-08-25T19:47:48.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3750K .......... .......... .......... .......... .......... 2% 55.5M 9s 2024-08-25T19:47:48.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3800K .......... .......... .......... .......... .......... 2% 25.7M 9s 2024-08-25T19:47:48.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3850K .......... .......... .......... .......... .......... 2% 29.2M 9s 2024-08-25T19:47:48.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3900K .......... .......... .......... .......... .......... 2% 21.3M 9s 2024-08-25T19:47:48.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 3950K .......... .......... .......... .......... .......... 2% 15.7M 9s 2024-08-25T19:47:48.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4000K .......... .......... .......... .......... .......... 2% 25.7M 9s 2024-08-25T19:47:48.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4050K .......... .......... .......... .......... .......... 2% 74.2M 9s 2024-08-25T19:47:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4100K .......... .......... .......... .......... .......... 2% 20.5M 8s 2024-08-25T19:47:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4150K .......... .......... .......... .......... .......... 2% 24.2M 8s 2024-08-25T19:47:48.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4200K .......... .......... .......... .......... .......... 2% 28.6M 8s 2024-08-25T19:47:48.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4250K .......... .......... .......... .......... .......... 2% 54.7M 8s 2024-08-25T19:47:48.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4300K .......... .......... .......... .......... .......... 2% 37.3M 8s 2024-08-25T19:47:48.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4350K .......... .......... .......... .......... .......... 2% 17.3M 8s 2024-08-25T19:47:48.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4400K .......... .......... .......... .......... .......... 2% 33.4M 8s 2024-08-25T19:47:48.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4450K .......... .......... .......... .......... .......... 2% 23.8M 8s 2024-08-25T19:47:48.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4500K .......... .......... .......... .......... .......... 2% 22.6M 8s 2024-08-25T19:47:48.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4550K .......... .......... .......... .......... .......... 2% 49.5M 8s 2024-08-25T19:47:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4600K .......... .......... .......... .......... .......... 2% 16.9M 8s 2024-08-25T19:47:48.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4650K .......... .......... .......... .......... .......... 2% 39.7M 8s 2024-08-25T19:47:48.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4700K .......... .......... .......... .......... .......... 2% 23.5M 8s 2024-08-25T19:47:48.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4750K .......... .......... .......... .......... .......... 2% 26.7M 8s 2024-08-25T19:47:48.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4800K .......... .......... .......... .......... .......... 2% 44.1M 8s 2024-08-25T19:47:48.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4850K .......... .......... .......... .......... .......... 2% 25.0M 8s 2024-08-25T19:47:48.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4900K .......... .......... .......... .......... .......... 2% 38.7M 8s 2024-08-25T19:47:48.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 4950K .......... .......... .......... .......... .......... 2% 24.3M 8s 2024-08-25T19:47:48.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5000K .......... .......... .......... .......... .......... 2% 22.0M 8s 2024-08-25T19:47:48.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5050K .......... .......... .......... .......... .......... 3% 43.2M 8s 2024-08-25T19:47:48.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5100K .......... .......... .......... .......... .......... 3% 26.8M 8s 2024-08-25T19:47:48.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5150K .......... .......... .......... .......... .......... 3% 22.5M 8s 2024-08-25T19:47:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5200K .......... .......... .......... .......... .......... 3% 36.1M 8s 2024-08-25T19:47:48.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5250K .......... .......... .......... .......... .......... 3% 25.2M 8s 2024-08-25T19:47:48.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5300K .......... .......... .......... .......... .......... 3% 38.5M 8s 2024-08-25T19:47:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5350K .......... .......... .......... .......... .......... 3% 27.1M 8s 2024-08-25T19:47:48.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5400K .......... .......... .......... .......... .......... 3% 28.0M 8s 2024-08-25T19:47:48.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5450K .......... .......... .......... .......... .......... 3% 24.0M 8s 2024-08-25T19:47:48.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5500K .......... .......... .......... .......... .......... 3% 41.5M 8s 2024-08-25T19:47:48.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5550K .......... .......... .......... .......... .......... 3% 16.0M 8s 2024-08-25T19:47:48.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5600K .......... .......... .......... .......... .......... 3% 53.1M 8s 2024-08-25T19:47:48.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5650K .......... .......... .......... .......... .......... 3% 23.2M 8s 2024-08-25T19:47:48.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5700K .......... .......... .......... .......... .......... 3% 33.1M 8s 2024-08-25T19:47:48.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5750K .......... .......... .......... .......... .......... 3% 23.2M 8s 2024-08-25T19:47:48.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5800K .......... .......... .......... .......... .......... 3% 56.4M 8s 2024-08-25T19:47:48.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5850K .......... .......... .......... .......... .......... 3% 21.8M 8s 2024-08-25T19:47:48.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5900K .......... .......... .......... .......... .......... 3% 39.3M 8s 2024-08-25T19:47:48.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 5950K .......... .......... .......... .......... .......... 3% 15.5M 8s 2024-08-25T19:47:48.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6000K .......... .......... .......... .......... .......... 3% 24.5M 8s 2024-08-25T19:47:48.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6050K .......... .......... .......... .......... .......... 3% 49.4M 8s 2024-08-25T19:47:48.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6100K .......... .......... .......... .......... .......... 3% 24.1M 8s 2024-08-25T19:47:48.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6150K .......... .......... .......... .......... .......... 3% 18.3M 8s 2024-08-25T19:47:48.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6200K .......... .......... .......... .......... .......... 3% 42.8M 8s 2024-08-25T19:47:48.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6250K .......... .......... .......... .......... .......... 3% 24.1M 7s 2024-08-25T19:47:48.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6300K .......... .......... .......... .......... .......... 3% 42.8M 7s 2024-08-25T19:47:48.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6350K .......... .......... .......... .......... .......... 3% 26.5M 7s 2024-08-25T19:47:48.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6400K .......... .......... .......... .......... .......... 3% 24.7M 7s 2024-08-25T19:47:48.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6450K .......... .......... .......... .......... .......... 3% 24.5M 7s 2024-08-25T19:47:48.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6500K .......... .......... .......... .......... .......... 3% 37.2M 7s 2024-08-25T19:47:48.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6550K .......... .......... .......... .......... .......... 3% 37.5M 7s 2024-08-25T19:47:48.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6600K .......... .......... .......... .......... .......... 3% 16.9M 7s 2024-08-25T19:47:48.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6650K .......... .......... .......... .......... .......... 3% 37.8M 7s 2024-08-25T19:47:48.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6700K .......... .......... .......... .......... .......... 4% 22.2M 7s 2024-08-25T19:47:48.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6750K .......... .......... .......... .......... .......... 4% 27.9M 7s 2024-08-25T19:47:48.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6800K .......... .......... .......... .......... .......... 4% 20.5M 7s 2024-08-25T19:47:48.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6850K .......... .......... .......... .......... .......... 4% 50.3M 7s 2024-08-25T19:47:48.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6900K .......... .......... .......... .......... .......... 4% 24.5M 7s 2024-08-25T19:47:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 6950K .......... .......... .......... .......... .......... 4% 38.8M 7s 2024-08-25T19:47:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7000K .......... .......... .......... .......... .......... 4% 24.7M 7s 2024-08-25T19:47:48.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7050K .......... .......... .......... .......... .......... 4% 17.5M 7s 2024-08-25T19:47:48.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7100K .......... .......... .......... .......... .......... 4% 26.5M 7s 2024-08-25T19:47:48.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7150K .......... .......... .......... .......... .......... 4% 23.9M 7s 2024-08-25T19:47:48.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7200K .......... .......... .......... .......... .......... 4% 33.9M 7s 2024-08-25T19:47:48.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7250K .......... .......... .......... .......... .......... 4% 23.1M 7s 2024-08-25T19:47:48.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7300K .......... .......... .......... .......... .......... 4% 23.6M 7s 2024-08-25T19:47:48.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7350K .......... .......... .......... .......... .......... 4% 20.3M 7s 2024-08-25T19:47:48.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7400K .......... .......... .......... .......... .......... 4% 25.3M 7s 2024-08-25T19:47:48.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7450K .......... .......... .......... .......... .......... 4% 30.4M 7s 2024-08-25T19:47:48.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7500K .......... .......... .......... .......... .......... 4% 46.1M 7s 2024-08-25T19:47:48.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7550K .......... .......... .......... .......... .......... 4% 10.4M 7s 2024-08-25T19:47:48.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7600K .......... .......... .......... .......... .......... 4% 52.5M 7s 2024-08-25T19:47:48.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7650K .......... .......... .......... .......... .......... 4% 24.5M 7s 2024-08-25T19:47:48.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7700K .......... .......... .......... .......... .......... 4% 28.2M 7s 2024-08-25T19:47:48.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7750K .......... .......... .......... .......... .......... 4% 29.8M 7s 2024-08-25T19:47:48.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7800K .......... .......... .......... .......... .......... 4% 14.8M 7s 2024-08-25T19:47:48.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7850K .......... .......... .......... .......... .......... 4% 52.0M 7s 2024-08-25T19:47:48.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7900K .......... .......... .......... .......... .......... 4% 30.1M 7s 2024-08-25T19:47:48.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 7950K .......... .......... .......... .......... .......... 4% 19.2M 7s 2024-08-25T19:47:48.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8000K .......... .......... .......... .......... .......... 4% 16.4M 7s 2024-08-25T19:47:48.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8050K .......... .......... .......... .......... .......... 4% 28.3M 7s 2024-08-25T19:47:48.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8100K .......... .......... .......... .......... .......... 4% 48.2M 7s 2024-08-25T19:47:48.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8150K .......... .......... .......... .......... .......... 4% 24.4M 7s 2024-08-25T19:47:48.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8200K .......... .......... .......... .......... .......... 4% 24.8M 7s 2024-08-25T19:47:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8250K .......... .......... .......... .......... .......... 4% 32.0M 7s 2024-08-25T19:47:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8300K .......... .......... .......... .......... .......... 4% 56.1M 7s 2024-08-25T19:47:48.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8350K .......... .......... .......... .......... .......... 4% 22.2M 7s 2024-08-25T19:47:48.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8400K .......... .......... .......... .......... .......... 5% 26.6M 7s 2024-08-25T19:47:48.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8450K .......... .......... .......... .......... .......... 5% 36.1M 7s 2024-08-25T19:47:48.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8500K .......... .......... .......... .......... .......... 5% 23.6M 7s 2024-08-25T19:47:48.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8550K .......... .......... .......... .......... .......... 5% 19.1M 7s 2024-08-25T19:47:48.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8600K .......... .......... .......... .......... .......... 5% 47.5M 7s 2024-08-25T19:47:48.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8650K .......... .......... .......... .......... .......... 5% 22.4M 7s 2024-08-25T19:47:48.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8700K .......... .......... .......... .......... .......... 5% 26.6M 7s 2024-08-25T19:47:48.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8750K .......... .......... .......... .......... .......... 5% 30.8M 7s 2024-08-25T19:47:48.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8800K .......... .......... .......... .......... .......... 5% 27.7M 7s 2024-08-25T19:47:48.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8850K .......... .......... .......... .......... .......... 5% 46.6M 7s 2024-08-25T19:47:48.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8900K .......... .......... .......... .......... .......... 5% 24.3M 7s 2024-08-25T19:47:48.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 8950K .......... .......... .......... .......... .......... 5% 37.9M 7s 2024-08-25T19:47:48.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9000K .......... .......... .......... .......... .......... 5% 23.9M 7s 2024-08-25T19:47:48.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9050K .......... .......... .......... .......... .......... 5% 21.4M 7s 2024-08-25T19:47:48.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9100K .......... .......... .......... .......... .......... 5% 59.0M 7s 2024-08-25T19:47:48.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9150K .......... .......... .......... .......... .......... 5% 19.9M 7s 2024-08-25T19:47:48.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9200K .......... .......... .......... .......... .......... 5% 50.5M 7s 2024-08-25T19:47:48.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9250K .......... .......... .......... .......... .......... 5% 19.3M 7s 2024-08-25T19:47:48.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9300K .......... .......... .......... .......... .......... 5% 25.8M 7s 2024-08-25T19:47:48.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9350K .......... .......... .......... .......... .......... 5% 43.7M 7s 2024-08-25T19:47:48.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9400K .......... .......... .......... .......... .......... 5% 22.2M 7s 2024-08-25T19:47:48.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9450K .......... .......... .......... .......... .......... 5% 28.3M 7s 2024-08-25T19:47:48.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9500K .......... .......... .......... .......... .......... 5% 25.4M 7s 2024-08-25T19:47:48.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9550K .......... .......... .......... .......... .......... 5% 21.0M 7s 2024-08-25T19:47:48.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9600K .......... .......... .......... .......... .......... 5% 26.7M 7s 2024-08-25T19:47:48.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9650K .......... .......... .......... .......... .......... 5% 45.0M 7s 2024-08-25T19:47:48.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9700K .......... .......... .......... .......... .......... 5% 21.1M 7s 2024-08-25T19:47:48.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9750K .......... .......... .......... .......... .......... 5% 40.9M 7s 2024-08-25T19:47:48.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9800K .......... .......... .......... .......... .......... 5% 18.4M 7s 2024-08-25T19:47:48.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9850K .......... .......... .......... .......... .......... 5% 50.2M 7s 2024-08-25T19:47:48.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9900K .......... .......... .......... .......... .......... 5% 24.6M 7s 2024-08-25T19:47:48.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 9950K .......... .......... .......... .......... .......... 5% 24.1M 7s 2024-08-25T19:47:48.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10000K .......... .......... .......... .......... .......... 5% 40.3M 7s 2024-08-25T19:47:48.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10050K .......... .......... .......... .......... .......... 5% 23.6M 7s 2024-08-25T19:47:48.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10100K .......... .......... .......... .......... .......... 6% 23.4M 7s 2024-08-25T19:47:48.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10150K .......... .......... .......... .......... .......... 6% 36.7M 7s 2024-08-25T19:47:48.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10200K .......... .......... .......... .......... .......... 6% 33.2M 7s 2024-08-25T19:47:48.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10250K .......... .......... .......... .......... .......... 6% 33.1M 7s 2024-08-25T19:47:48.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10300K .......... .......... .......... .......... .......... 6% 23.8M 7s 2024-08-25T19:47:48.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10350K .......... .......... .......... .......... .......... 6% 24.2M 7s 2024-08-25T19:47:48.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10400K .......... .......... .......... .......... .......... 6% 41.8M 7s 2024-08-25T19:47:48.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10450K .......... .......... .......... .......... .......... 6% 25.1M 7s 2024-08-25T19:47:48.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10500K .......... .......... .......... .......... .......... 6% 36.5M 7s 2024-08-25T19:47:48.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10550K .......... .......... .......... .......... .......... 6% 23.6M 7s 2024-08-25T19:47:48.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10600K .......... .......... .......... .......... .......... 6% 41.7M 7s 2024-08-25T19:47:48.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10650K .......... .......... .......... .......... .......... 6% 23.4M 7s 2024-08-25T19:47:48.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10700K .......... .......... .......... .......... .......... 6% 23.1M 7s 2024-08-25T19:47:48.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10750K .......... .......... .......... .......... .......... 6% 24.1M 7s 2024-08-25T19:47:48.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10800K .......... .......... .......... .......... .......... 6% 41.7M 7s 2024-08-25T19:47:48.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10850K .......... .......... .......... .......... .......... 6% 23.5M 7s 2024-08-25T19:47:48.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10900K .......... .......... .......... .......... .......... 6% 49.3M 7s 2024-08-25T19:47:48.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 10950K .......... .......... .......... .......... .......... 6% 23.1M 7s 2024-08-25T19:47:48.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11000K .......... .......... .......... .......... .......... 6% 42.7M 7s 2024-08-25T19:47:48.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11050K .......... .......... .......... .......... .......... 6% 24.0M 7s 2024-08-25T19:47:48.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11100K .......... .......... .......... .......... .......... 6% 45.9M 7s 2024-08-25T19:47:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11150K .......... .......... .......... .......... .......... 6% 22.2M 7s 2024-08-25T19:47:48.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11200K .......... .......... .......... .......... .......... 6% 25.1M 7s 2024-08-25T19:47:48.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11250K .......... .......... .......... .......... .......... 6% 35.2M 7s 2024-08-25T19:47:48.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11300K .......... .......... .......... .......... .......... 6% 23.7M 7s 2024-08-25T19:47:48.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11350K .......... .......... .......... .......... .......... 6% 24.4M 7s 2024-08-25T19:47:48.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11400K .......... .......... .......... .......... .......... 6% 41.3M 7s 2024-08-25T19:47:48.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11450K .......... .......... .......... .......... .......... 6% 36.8M 7s 2024-08-25T19:47:48.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11500K .......... .......... .......... .......... .......... 6% 16.8M 7s 2024-08-25T19:47:48.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11550K .......... .......... .......... .......... .......... 6% 61.6M 7s 2024-08-25T19:47:48.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11600K .......... .......... .......... .......... .......... 6% 22.6M 7s 2024-08-25T19:47:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11650K .......... .......... .......... .......... .......... 6% 50.3M 6s 2024-08-25T19:47:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11700K .......... .......... .......... .......... .......... 6% 23.4M 6s 2024-08-25T19:47:48.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11750K .......... .......... .......... .......... .......... 7% 37.0M 6s 2024-08-25T19:47:48.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11800K .......... .......... .......... .......... .......... 7% 23.9M 6s 2024-08-25T19:47:48.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11850K .......... .......... .......... .......... .......... 7% 24.1M 6s 2024-08-25T19:47:48.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11900K .......... .......... .......... .......... .......... 7% 35.8M 6s 2024-08-25T19:47:48.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 11950K .......... .......... .......... .......... .......... 7% 25.8M 6s 2024-08-25T19:47:48.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12000K .......... .......... .......... .......... .......... 7% 23.7M 6s 2024-08-25T19:47:48.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12050K .......... .......... .......... .......... .......... 7% 39.8M 6s 2024-08-25T19:47:48.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12100K .......... .......... .......... .......... .......... 7% 25.7M 6s 2024-08-25T19:47:48.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12150K .......... .......... .......... .......... .......... 7% 44.9M 6s 2024-08-25T19:47:48.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12200K .......... .......... .......... .......... .......... 7% 24.6M 6s 2024-08-25T19:47:48.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12250K .......... .......... .......... .......... .......... 7% 17.7M 6s 2024-08-25T19:47:48.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12300K .......... .......... .......... .......... .......... 7% 30.0M 6s 2024-08-25T19:47:48.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12350K .......... .......... .......... .......... .......... 7% 19.5M 6s 2024-08-25T19:47:48.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12400K .......... .......... .......... .......... .......... 7% 25.1M 6s 2024-08-25T19:47:48.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12450K .......... .......... .......... .......... .......... 7% 47.3M 6s 2024-08-25T19:47:48.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12500K .......... .......... .......... .......... .......... 7% 20.2M 6s 2024-08-25T19:47:48.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12550K .......... .......... .......... .......... .......... 7% 32.2M 6s 2024-08-25T19:47:48.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12600K .......... .......... .......... .......... .......... 7% 25.8M 6s 2024-08-25T19:47:48.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12650K .......... .......... .......... .......... .......... 7% 50.4M 6s 2024-08-25T19:47:48.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12700K .......... .......... .......... .......... .......... 7% 16.0M 6s 2024-08-25T19:47:48.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12750K .......... .......... .......... .......... .......... 7% 19.9M 6s 2024-08-25T19:47:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12800K .......... .......... .......... .......... .......... 7% 35.6M 6s 2024-08-25T19:47:48.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12850K .......... .......... .......... .......... .......... 7% 27.1M 6s 2024-08-25T19:47:48.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12900K .......... .......... .......... .......... .......... 7% 21.5M 6s 2024-08-25T19:47:48.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 12950K .......... .......... .......... .......... .......... 7% 57.1M 6s 2024-08-25T19:47:48.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13000K .......... .......... .......... .......... .......... 7% 26.1M 6s 2024-08-25T19:47:48.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13050K .......... .......... .......... .......... .......... 7% 20.1M 6s 2024-08-25T19:47:48.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13100K .......... .......... .......... .......... .......... 7% 28.5M 6s 2024-08-25T19:47:48.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13150K .......... .......... .......... .......... .......... 7% 21.3M 6s 2024-08-25T19:47:48.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13200K .......... .......... .......... .......... .......... 7% 50.3M 6s 2024-08-25T19:47:48.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13250K .......... .......... .......... .......... .......... 7% 19.8M 6s 2024-08-25T19:47:48.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13300K .......... .......... .......... .......... .......... 7% 44.5M 6s 2024-08-25T19:47:48.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13350K .......... .......... .......... .......... .......... 7% 20.1M 6s 2024-08-25T19:47:48.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13400K .......... .......... .......... .......... .......... 7% 65.1M 6s 2024-08-25T19:47:48.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13450K .......... .......... .......... .......... .......... 8% 18.8M 6s 2024-08-25T19:47:48.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13500K .......... .......... .......... .......... .......... 8% 24.3M 6s 2024-08-25T19:47:48.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13550K .......... .......... .......... .......... .......... 8% 27.6M 6s 2024-08-25T19:47:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13600K .......... .......... .......... .......... .......... 8% 23.9M 6s 2024-08-25T19:47:48.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13650K .......... .......... .......... .......... .......... 8% 25.5M 6s 2024-08-25T19:47:48.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13700K .......... .......... .......... .......... .......... 8% 57.3M 6s 2024-08-25T19:47:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13750K .......... .......... .......... .......... .......... 8% 19.7M 6s 2024-08-25T19:47:49.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13800K .......... .......... .......... .......... .......... 8% 19.2M 6s 2024-08-25T19:47:49.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13850K .......... .......... .......... .......... .......... 8% 23.7M 6s 2024-08-25T19:47:49.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13900K .......... .......... .......... .......... .......... 8% 55.8M 6s 2024-08-25T19:47:49.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 13950K .......... .......... .......... .......... .......... 8% 22.8M 6s 2024-08-25T19:47:49.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14000K .......... .......... .......... .......... .......... 8% 22.8M 6s 2024-08-25T19:47:49.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14050K .......... .......... .......... .......... .......... 8% 38.5M 6s 2024-08-25T19:47:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14100K .......... .......... .......... .......... .......... 8% 20.9M 6s 2024-08-25T19:47:49.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14150K .......... .......... .......... .......... .......... 8% 26.7M 6s 2024-08-25T19:47:49.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14200K .......... .......... .......... .......... .......... 8% 43.0M 6s 2024-08-25T19:47:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14250K .......... .......... .......... .......... .......... 8% 24.2M 6s 2024-08-25T19:47:49.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14300K .......... .......... .......... .......... .......... 8% 33.1M 6s 2024-08-25T19:47:49.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14350K .......... .......... .......... .......... .......... 8% 14.8M 6s 2024-08-25T19:47:49.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14400K .......... .......... .......... .......... .......... 8% 24.5M 6s 2024-08-25T19:47:49.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14450K .......... .......... .......... .......... .......... 8% 43.5M 6s 2024-08-25T19:47:49.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14500K .......... .......... .......... .......... .......... 8% 25.5M 6s 2024-08-25T19:47:49.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14550K .......... .......... .......... .......... .......... 8% 40.2M 6s 2024-08-25T19:47:49.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14600K .......... .......... .......... .......... .......... 8% 24.8M 6s 2024-08-25T19:47:49.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14650K .......... .......... .......... .......... .......... 8% 21.1M 6s 2024-08-25T19:47:49.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14700K .......... .......... .......... .......... .......... 8% 58.6M 6s 2024-08-25T19:47:49.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14750K .......... .......... .......... .......... .......... 8% 17.9M 6s 2024-08-25T19:47:49.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14800K .......... .......... .......... .......... .......... 8% 35.3M 6s 2024-08-25T19:47:49.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14850K .......... .......... .......... .......... .......... 8% 28.7M 6s 2024-08-25T19:47:49.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14900K .......... .......... .......... .......... .......... 8% 17.1M 6s 2024-08-25T19:47:49.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 14950K .......... .......... .......... .......... .......... 8% 45.6M 6s 2024-08-25T19:47:49.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15000K .......... .......... .......... .......... .......... 8% 24.4M 6s 2024-08-25T19:47:49.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15050K .......... .......... .......... .......... .......... 8% 41.8M 6s 2024-08-25T19:47:49.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15100K .......... .......... .......... .......... .......... 8% 24.4M 6s 2024-08-25T19:47:49.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15150K .......... .......... .......... .......... .......... 9% 22.9M 6s 2024-08-25T19:47:49.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15200K .......... .......... .......... .......... .......... 9% 24.0M 6s 2024-08-25T19:47:49.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15250K .......... .......... .......... .......... .......... 9% 48.4M 6s 2024-08-25T19:47:49.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15300K .......... .......... .......... .......... .......... 9% 28.0M 6s 2024-08-25T19:47:49.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15350K .......... .......... .......... .......... .......... 9% 30.0M 6s 2024-08-25T19:47:49.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15400K .......... .......... .......... .......... .......... 9% 24.0M 6s 2024-08-25T19:47:49.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15450K .......... .......... .......... .......... .......... 9% 48.9M 6s 2024-08-25T19:47:49.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15500K .......... .......... .......... .......... .......... 9% 25.5M 6s 2024-08-25T19:47:49.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15550K .......... .......... .......... .......... .......... 9% 22.7M 6s 2024-08-25T19:47:49.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15600K .......... .......... .......... .......... .......... 9% 39.5M 6s 2024-08-25T19:47:49.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15650K .......... .......... .......... .......... .......... 9% 24.1M 6s 2024-08-25T19:47:49.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15700K .......... .......... .......... .......... .......... 9% 38.4M 6s 2024-08-25T19:47:49.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15750K .......... .......... .......... .......... .......... 9% 26.6M 6s 2024-08-25T19:47:49.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15800K .......... .......... .......... .......... .......... 9% 32.6M 6s 2024-08-25T19:47:49.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15850K .......... .......... .......... .......... .......... 9% 22.6M 6s 2024-08-25T19:47:49.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15900K .......... .......... .......... .......... .......... 9% 24.7M 6s 2024-08-25T19:47:49.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 15950K .......... .......... .......... .......... .......... 9% 15.8M 6s 2024-08-25T19:47:49.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16000K .......... .......... .......... .......... .......... 9% 47.4M 6s 2024-08-25T19:47:49.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16050K .......... .......... .......... .......... .......... 9% 24.7M 6s 2024-08-25T19:47:49.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16100K .......... .......... .......... .......... .......... 9% 28.2M 6s 2024-08-25T19:47:49.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16150K .......... .......... .......... .......... .......... 9% 24.7M 6s 2024-08-25T19:47:49.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16200K .......... .......... .......... .......... .......... 9% 39.8M 6s 2024-08-25T19:47:49.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16250K .......... .......... .......... .......... .......... 9% 22.0M 6s 2024-08-25T19:47:49.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16300K .......... .......... .......... .......... .......... 9% 40.0M 6s 2024-08-25T19:47:49.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16350K .......... .......... .......... .......... .......... 9% 16.4M 6s 2024-08-25T19:47:49.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16400K .......... .......... .......... .......... .......... 9% 42.0M 6s 2024-08-25T19:47:49.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16450K .......... .......... .......... .......... .......... 9% 19.2M 6s 2024-08-25T19:47:49.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16500K .......... .......... .......... .......... .......... 9% 42.1M 6s 2024-08-25T19:47:49.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16550K .......... .......... .......... .......... .......... 9% 23.5M 6s 2024-08-25T19:47:49.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16600K .......... .......... .......... .......... .......... 9% 24.8M 6s 2024-08-25T19:47:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16650K .......... .......... .......... .......... .......... 9% 24.6M 6s 2024-08-25T19:47:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16700K .......... .......... .......... .......... .......... 9% 146M 6s 2024-08-25T19:47:49.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16750K .......... .......... .......... .......... .......... 9% 22.0M 6s 2024-08-25T19:47:49.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16800K .......... .......... .......... .......... .......... 9% 17.9M 6s 2024-08-25T19:47:49.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16850K .......... .......... .......... .......... .......... 10% 28.6M 6s 2024-08-25T19:47:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16900K .......... .......... .......... .......... .......... 10% 27.1M 6s 2024-08-25T19:47:49.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 16950K .......... .......... .......... .......... .......... 10% 26.0M 6s 2024-08-25T19:47:49.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17000K .......... .......... .......... .......... .......... 10% 42.6M 6s 2024-08-25T19:47:49.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17050K .......... .......... .......... .......... .......... 10% 24.7M 6s 2024-08-25T19:47:49.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17100K .......... .......... .......... .......... .......... 10% 37.4M 6s 2024-08-25T19:47:49.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17150K .......... .......... .......... .......... .......... 10% 24.4M 6s 2024-08-25T19:47:49.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17200K .......... .......... .......... .......... .......... 10% 14.1M 6s 2024-08-25T19:47:49.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17250K .......... .......... .......... .......... .......... 10% 44.7M 6s 2024-08-25T19:47:49.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17300K .......... .......... .......... .......... .......... 10% 27.2M 6s 2024-08-25T19:47:49.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17350K .......... .......... .......... .......... .......... 10% 27.0M 6s 2024-08-25T19:47:49.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17400K .......... .......... .......... .......... .......... 10% 23.5M 6s 2024-08-25T19:47:49.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17450K .......... .......... .......... .......... .......... 10% 21.8M 6s 2024-08-25T19:47:49.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17500K .......... .......... .......... .......... .......... 10% 49.4M 6s 2024-08-25T19:47:49.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17550K .......... .......... .......... .......... .......... 10% 25.3M 6s 2024-08-25T19:47:49.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17600K .......... .......... .......... .......... .......... 10% 19.4M 6s 2024-08-25T19:47:49.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17650K .......... .......... .......... .......... .......... 10% 34.6M 6s 2024-08-25T19:47:49.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17700K .......... .......... .......... .......... .......... 10% 25.2M 6s 2024-08-25T19:47:49.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17750K .......... .......... .......... .......... .......... 10% 39.8M 6s 2024-08-25T19:47:49.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17800K .......... .......... .......... .......... .......... 10% 19.5M 6s 2024-08-25T19:47:49.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17850K .......... .......... .......... .......... .......... 10% 15.8M 6s 2024-08-25T19:47:49.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17900K .......... .......... .......... .......... .......... 10% 43.0M 6s 2024-08-25T19:47:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 17950K .......... .......... .......... .......... .......... 10% 28.7M 6s 2024-08-25T19:47:49.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18000K .......... .......... .......... .......... .......... 10% 19.4M 6s 2024-08-25T19:47:49.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18050K .......... .......... .......... .......... .......... 10% 51.2M 6s 2024-08-25T19:47:49.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18100K .......... .......... .......... .......... .......... 10% 18.2M 6s 2024-08-25T19:47:49.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18150K .......... .......... .......... .......... .......... 10% 40.3M 6s 2024-08-25T19:47:49.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18200K .......... .......... .......... .......... .......... 10% 35.1M 6s 2024-08-25T19:47:49.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18250K .......... .......... .......... .......... .......... 10% 31.2M 6s 2024-08-25T19:47:49.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18300K .......... .......... .......... .......... .......... 10% 18.4M 6s 2024-08-25T19:47:49.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18350K .......... .......... .......... .......... .......... 10% 23.6M 6s 2024-08-25T19:47:49.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18400K .......... .......... .......... .......... .......... 10% 36.8M 6s 2024-08-25T19:47:49.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18450K .......... .......... .......... .......... .......... 10% 18.6M 6s 2024-08-25T19:47:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18500K .......... .......... .......... .......... .......... 11% 626K 7s 2024-08-25T19:47:49.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18550K .......... .......... .......... .......... .......... 11% 174M 7s 2024-08-25T19:47:49.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18600K .......... .......... .......... .......... .......... 11% 181M 7s 2024-08-25T19:47:49.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18650K .......... .......... .......... .......... .......... 11% 203M 7s 2024-08-25T19:47:49.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18700K .......... .......... .......... .......... .......... 11% 198M 7s 2024-08-25T19:47:49.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18750K .......... .......... .......... .......... .......... 11% 168M 6s 2024-08-25T19:47:49.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18800K .......... .......... .......... .......... .......... 11% 193M 6s 2024-08-25T19:47:49.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18850K .......... .......... .......... .......... .......... 11% 205M 6s 2024-08-25T19:47:49.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18900K .......... .......... .......... .......... .......... 11% 8.15M 6s 2024-08-25T19:47:49.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 18950K .......... .......... .......... .......... .......... 11% 170M 6s 2024-08-25T19:47:49.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19000K .......... .......... .......... .......... .......... 11% 183M 6s 2024-08-25T19:47:49.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19050K .......... .......... .......... .......... .......... 11% 194M 6s 2024-08-25T19:47:49.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19100K .......... .......... .......... .......... .......... 11% 202M 6s 2024-08-25T19:47:49.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19150K .......... .......... .......... .......... .......... 11% 167M 6s 2024-08-25T19:47:49.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19200K .......... .......... .......... .......... .......... 11% 189M 6s 2024-08-25T19:47:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19250K .......... .......... .......... .......... .......... 11% 73.1M 6s 2024-08-25T19:47:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19300K .......... .......... .......... .......... .......... 11% 175M 6s 2024-08-25T19:47:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19350K .......... .......... .......... .......... .......... 11% 176M 6s 2024-08-25T19:47:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19400K .......... .......... .......... .......... .......... 11% 199M 6s 2024-08-25T19:47:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19450K .......... .......... .......... .......... .......... 11% 198M 6s 2024-08-25T19:47:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19500K .......... .......... .......... .......... .......... 11% 180M 6s 2024-08-25T19:47:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19550K .......... .......... .......... .......... .......... 11% 167M 6s 2024-08-25T19:47:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19600K .......... .......... .......... .......... .......... 11% 203M 6s 2024-08-25T19:47:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19650K .......... .......... .......... .......... .......... 11% 197M 6s 2024-08-25T19:47:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19700K .......... .......... .......... .......... .......... 11% 191M 6s 2024-08-25T19:47:49.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19750K .......... .......... .......... .......... .......... 11% 2.52M 6s 2024-08-25T19:47:49.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19800K .......... .......... .......... .......... .......... 11% 160M 6s 2024-08-25T19:47:49.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19850K .......... .......... .......... .......... .......... 11% 183M 6s 2024-08-25T19:47:49.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19900K .......... .......... .......... .......... .......... 11% 195M 6s 2024-08-25T19:47:49.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 19950K .......... .......... .......... .......... .......... 11% 167M 6s 2024-08-25T19:47:49.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20000K .......... .......... .......... .......... .......... 11% 203M 6s 2024-08-25T19:47:49.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20050K .......... .......... .......... .......... .......... 11% 185M 6s 2024-08-25T19:47:49.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20100K .......... .......... .......... .......... .......... 11% 202M 6s 2024-08-25T19:47:49.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20150K .......... .......... .......... .......... .......... 11% 152M 6s 2024-08-25T19:47:49.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20200K .......... .......... .......... .......... .......... 12% 199M 6s 2024-08-25T19:47:49.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20250K .......... .......... .......... .......... .......... 12% 192M 6s 2024-08-25T19:47:49.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20300K .......... .......... .......... .......... .......... 12% 196M 6s 2024-08-25T19:47:49.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20350K .......... .......... .......... .......... .......... 12% 170M 6s 2024-08-25T19:47:49.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20400K .......... .......... .......... .......... .......... 12% 198M 6s 2024-08-25T19:47:49.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20450K .......... .......... .......... .......... .......... 12% 192M 6s 2024-08-25T19:47:49.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20500K .......... .......... .......... .......... .......... 12% 205M 6s 2024-08-25T19:47:49.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20550K .......... .......... .......... .......... .......... 12% 176M 6s 2024-08-25T19:47:49.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20600K .......... .......... .......... .......... .......... 12% 201M 6s 2024-08-25T19:47:49.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20650K .......... .......... .......... .......... .......... 12% 186M 6s 2024-08-25T19:47:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20700K .......... .......... .......... .......... .......... 12% 184M 6s 2024-08-25T19:47:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20750K .......... .......... .......... .......... .......... 12% 156M 6s 2024-08-25T19:47:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20800K .......... .......... .......... .......... .......... 12% 185M 6s 2024-08-25T19:47:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20850K .......... .......... .......... .......... .......... 12% 201M 6s 2024-08-25T19:47:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20900K .......... .......... .......... .......... .......... 12% 197M 6s 2024-08-25T19:47:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 20950K .......... .......... .......... .......... .......... 12% 182M 6s 2024-08-25T19:47:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21000K .......... .......... .......... .......... .......... 12% 173M 6s 2024-08-25T19:47:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21050K .......... .......... .......... .......... .......... 12% 199M 6s 2024-08-25T19:47:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21100K .......... .......... .......... .......... .......... 12% 173M 6s 2024-08-25T19:47:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21150K .......... .......... .......... .......... .......... 12% 169M 6s 2024-08-25T19:47:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21200K .......... .......... .......... .......... .......... 12% 167M 6s 2024-08-25T19:47:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21250K .......... .......... .......... .......... .......... 12% 197M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21300K .......... .......... .......... .......... .......... 12% 195M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21350K .......... .......... .......... .......... .......... 12% 174M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21400K .......... .......... .......... .......... .......... 12% 181M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21450K .......... .......... .......... .......... .......... 12% 203M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21500K .......... .......... .......... .......... .......... 12% 198M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21550K .......... .......... .......... .......... .......... 12% 162M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21600K .......... .......... .......... .......... .......... 12% 204M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21650K .......... .......... .......... .......... .......... 12% 178M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21700K .......... .......... .......... .......... .......... 12% 25.8M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21750K .......... .......... .......... .......... .......... 12% 42.8M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21800K .......... .......... .......... .......... .......... 12% 17.5M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21850K .......... .......... .......... .......... .......... 12% 42.4M 6s 2024-08-25T19:47:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21900K .......... .......... .......... .......... .......... 13% 36.7M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 21950K .......... .......... .......... .......... .......... 13% 18.6M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22000K .......... .......... .......... .......... .......... 13% 37.4M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22050K .......... .......... .......... .......... .......... 13% 21.5M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22100K .......... .......... .......... .......... .......... 13% 21.2M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22150K .......... .......... .......... .......... .......... 13% 27.9M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22200K .......... .......... .......... .......... .......... 13% 29.2M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22250K .......... .......... .......... .......... .......... 13% 20.9M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22300K .......... .......... .......... .......... .......... 13% 91.6M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22350K .......... .......... .......... .......... .......... 13% 18.2M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22400K .......... .......... .......... .......... .......... 13% 40.2M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22450K .......... .......... .......... .......... .......... 13% 27.5M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22500K .......... .......... .......... .......... .......... 13% 21.7M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22550K .......... .......... .......... .......... .......... 13% 42.7M 6s 2024-08-25T19:47:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22600K .......... .......... .......... .......... .......... 13% 33.8M 6s 2024-08-25T19:47:49.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22650K .......... .......... .......... .......... .......... 13% 30.9M 6s 2024-08-25T19:47:49.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22700K .......... .......... .......... .......... .......... 13% 24.2M 6s 2024-08-25T19:47:49.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22750K .......... .......... .......... .......... .......... 13% 23.9M 6s 2024-08-25T19:47:49.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22800K .......... .......... .......... .......... .......... 13% 37.2M 6s 2024-08-25T19:47:49.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22850K .......... .......... .......... .......... .......... 13% 24.1M 6s 2024-08-25T19:47:49.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22900K .......... .......... .......... .......... .......... 13% 43.9M 6s 2024-08-25T19:47:49.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 22950K .......... .......... .......... .......... .......... 13% 23.5M 6s 2024-08-25T19:47:49.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23000K .......... .......... .......... .......... .......... 13% 44.2M 6s 2024-08-25T19:47:49.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23050K .......... .......... .......... .......... .......... 13% 23.5M 6s 2024-08-25T19:47:49.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23100K .......... .......... .......... .......... .......... 13% 44.8M 6s 2024-08-25T19:47:49.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23150K .......... .......... .......... .......... .......... 13% 22.0M 6s 2024-08-25T19:47:49.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23200K .......... .......... .......... .......... .......... 13% 18.6M 6s 2024-08-25T19:47:49.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23250K .......... .......... .......... .......... .......... 13% 20.7M 6s 2024-08-25T19:47:49.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23300K .......... .......... .......... .......... .......... 13% 24.9M 6s 2024-08-25T19:47:49.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23350K .......... .......... .......... .......... .......... 13% 22.7M 6s 2024-08-25T19:47:49.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23400K .......... .......... .......... .......... .......... 13% 144M 6s 2024-08-25T19:47:49.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23450K .......... .......... .......... .......... .......... 13% 24.0M 6s 2024-08-25T19:47:49.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23500K .......... .......... .......... .......... .......... 13% 18.1M 6s 2024-08-25T19:47:49.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23550K .......... .......... .......... .......... .......... 14% 19.1M 6s 2024-08-25T19:47:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23600K .......... .......... .......... .......... .......... 14% 26.0M 6s 2024-08-25T19:47:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23650K .......... .......... .......... .......... .......... 14% 94.0M 6s 2024-08-25T19:47:49.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23700K .......... .......... .......... .......... .......... 14% 21.3M 6s 2024-08-25T19:47:49.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23750K .......... .......... .......... .......... .......... 14% 39.7M 6s 2024-08-25T19:47:49.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23800K .......... .......... .......... .......... .......... 14% 19.5M 6s 2024-08-25T19:47:49.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23850K .......... .......... .......... .......... .......... 14% 15.3M 6s 2024-08-25T19:47:49.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23900K .......... .......... .......... .......... .......... 14% 156M 6s 2024-08-25T19:47:49.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 23950K .......... .......... .......... .......... .......... 14% 21.9M 6s 2024-08-25T19:47:49.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24000K .......... .......... .......... .......... .......... 14% 20.5M 6s 2024-08-25T19:47:49.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24050K .......... .......... .......... .......... .......... 14% 24.9M 6s 2024-08-25T19:47:49.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24100K .......... .......... .......... .......... .......... 14% 28.9M 6s 2024-08-25T19:47:49.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24150K .......... .......... .......... .......... .......... 14% 86.2M 6s 2024-08-25T19:47:49.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24200K .......... .......... .......... .......... .......... 14% 25.3M 6s 2024-08-25T19:47:49.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24250K .......... .......... .......... .......... .......... 14% 24.4M 6s 2024-08-25T19:47:49.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24300K .......... .......... .......... .......... .......... 14% 25.7M 6s 2024-08-25T19:47:49.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24350K .......... .......... .......... .......... .......... 14% 25.0M 6s 2024-08-25T19:47:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24400K .......... .......... .......... .......... .......... 14% 25.2M 6s 2024-08-25T19:47:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24450K .......... .......... .......... .......... .......... 14% 45.9M 6s 2024-08-25T19:47:49.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24500K .......... .......... .......... .......... .......... 14% 23.9M 6s 2024-08-25T19:47:49.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24550K .......... .......... .......... .......... .......... 14% 28.6M 6s 2024-08-25T19:47:49.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24600K .......... .......... .......... .......... .......... 14% 14.6M 6s 2024-08-25T19:47:49.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24650K .......... .......... .......... .......... .......... 14% 50.7M 6s 2024-08-25T19:47:49.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24700K .......... .......... .......... .......... .......... 14% 23.1M 6s 2024-08-25T19:47:49.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24750K .......... .......... .......... .......... .......... 14% 24.4M 6s 2024-08-25T19:47:49.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24800K .......... .......... .......... .......... .......... 14% 26.6M 6s 2024-08-25T19:47:49.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24850K .......... .......... .......... .......... .......... 14% 26.9M 6s 2024-08-25T19:47:49.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24900K .......... .......... .......... .......... .......... 14% 37.1M 6s 2024-08-25T19:47:49.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 24950K .......... .......... .......... .......... .......... 14% 17.8M 6s 2024-08-25T19:47:49.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25000K .......... .......... .......... .......... .......... 14% 24.5M 6s 2024-08-25T19:47:49.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25050K .......... .......... .......... .......... .......... 14% 39.4M 6s 2024-08-25T19:47:49.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25100K .......... .......... .......... .......... .......... 14% 24.6M 6s 2024-08-25T19:47:49.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25150K .......... .......... .......... .......... .......... 14% 24.3M 6s 2024-08-25T19:47:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25200K .......... .......... .......... .......... .......... 14% 43.5M 6s 2024-08-25T19:47:49.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25250K .......... .......... .......... .......... .......... 15% 19.6M 6s 2024-08-25T19:47:49.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25300K .......... .......... .......... .......... .......... 15% 42.4M 6s 2024-08-25T19:47:49.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25350K .......... .......... .......... .......... .......... 15% 23.6M 6s 2024-08-25T19:47:49.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25400K .......... .......... .......... .......... .......... 15% 44.1M 6s 2024-08-25T19:47:49.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25450K .......... .......... .......... .......... .......... 15% 16.5M 6s 2024-08-25T19:47:49.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25500K .......... .......... .......... .......... .......... 15% 35.6M 6s 2024-08-25T19:47:49.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25550K .......... .......... .......... .......... .......... 15% 21.3M 6s 2024-08-25T19:47:49.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25600K .......... .......... .......... .......... .......... 15% 23.8M 6s 2024-08-25T19:47:49.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25650K .......... .......... .......... .......... .......... 15% 26.7M 6s 2024-08-25T19:47:49.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25700K .......... .......... .......... .......... .......... 15% 29.8M 6s 2024-08-25T19:47:49.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25750K .......... .......... .......... .......... .......... 15% 25.3M 6s 2024-08-25T19:47:49.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25800K .......... .......... .......... .......... .......... 15% 33.5M 6s 2024-08-25T19:47:49.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25850K .......... .......... .......... .......... .......... 15% 27.6M 6s 2024-08-25T19:47:49.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25900K .......... .......... .......... .......... .......... 15% 48.0M 6s 2024-08-25T19:47:49.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 25950K .......... .......... .......... .......... .......... 15% 20.8M 6s 2024-08-25T19:47:49.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26000K .......... .......... .......... .......... .......... 15% 26.4M 6s 2024-08-25T19:47:49.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26050K .......... .......... .......... .......... .......... 15% 41.3M 6s 2024-08-25T19:47:49.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26100K .......... .......... .......... .......... .......... 15% 23.0M 6s 2024-08-25T19:47:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26150K .......... .......... .......... .......... .......... 15% 21.3M 6s 2024-08-25T19:47:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26200K .......... .......... .......... .......... .......... 15% 48.4M 6s 2024-08-25T19:47:49.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26250K .......... .......... .......... .......... .......... 15% 13.2M 6s 2024-08-25T19:47:49.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26300K .......... .......... .......... .......... .......... 15% 39.1M 6s 2024-08-25T19:47:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26350K .......... .......... .......... .......... .......... 15% 21.3M 6s 2024-08-25T19:47:49.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26400K .......... .......... .......... .......... .......... 15% 29.9M 6s 2024-08-25T19:47:49.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26450K .......... .......... .......... .......... .......... 15% 32.2M 6s 2024-08-25T19:47:49.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26500K .......... .......... .......... .......... .......... 15% 24.1M 6s 2024-08-25T19:47:49.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26550K .......... .......... .......... .......... .......... 15% 38.7M 6s 2024-08-25T19:47:49.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26600K .......... .......... .......... .......... .......... 15% 33.6M 6s 2024-08-25T19:47:49.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26650K .......... .......... .......... .......... .......... 15% 17.5M 6s 2024-08-25T19:47:49.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26700K .......... .......... .......... .......... .......... 15% 42.3M 6s 2024-08-25T19:47:49.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26750K .......... .......... .......... .......... .......... 15% 28.1M 6s 2024-08-25T19:47:49.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26800K .......... .......... .......... .......... .......... 15% 24.5M 6s 2024-08-25T19:47:49.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26850K .......... .......... .......... .......... .......... 15% 40.1M 5s 2024-08-25T19:47:49.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26900K .......... .......... .......... .......... .......... 15% 23.3M 5s 2024-08-25T19:47:49.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 26950K .......... .......... .......... .......... .......... 16% 51.9M 5s 2024-08-25T19:47:49.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27000K .......... .......... .......... .......... .......... 16% 22.7M 5s 2024-08-25T19:47:49.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27050K .......... .......... .......... .......... .......... 16% 43.1M 5s 2024-08-25T19:47:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27100K .......... .......... .......... .......... .......... 16% 23.4M 5s 2024-08-25T19:47:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27150K .......... .......... .......... .......... .......... 16% 24.9M 5s 2024-08-25T19:47:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27200K .......... .......... .......... .......... .......... 16% 22.3M 5s 2024-08-25T19:47:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27250K .......... .......... .......... .......... .......... 16% 33.7M 5s 2024-08-25T19:47:49.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27300K .......... .......... .......... .......... .......... 16% 24.6M 5s 2024-08-25T19:47:49.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27350K .......... .......... .......... .......... .......... 16% 41.5M 5s 2024-08-25T19:47:49.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27400K .......... .......... .......... .......... .......... 16% 16.2M 5s 2024-08-25T19:47:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27450K .......... .......... .......... .......... .......... 16% 43.3M 5s 2024-08-25T19:47:49.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27500K .......... .......... .......... .......... .......... 16% 38.5M 5s 2024-08-25T19:47:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27550K .......... .......... .......... .......... .......... 16% 17.7M 5s 2024-08-25T19:47:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27600K .......... .......... .......... .......... .......... 16% 43.0M 5s 2024-08-25T19:47:49.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27650K .......... .......... .......... .......... .......... 16% 21.8M 5s 2024-08-25T19:47:49.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27700K .......... .......... .......... .......... .......... 16% 44.0M 5s 2024-08-25T19:47:49.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27750K .......... .......... .......... .......... .......... 16% 22.5M 5s 2024-08-25T19:47:49.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27800K .......... .......... .......... .......... .......... 16% 21.1M 5s 2024-08-25T19:47:49.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27850K .......... .......... .......... .......... .......... 16% 27.8M 5s 2024-08-25T19:47:49.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27900K .......... .......... .......... .......... .......... 16% 24.1M 5s 2024-08-25T19:47:49.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 27950K .......... .......... .......... .......... .......... 16% 21.9M 5s 2024-08-25T19:47:49.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28000K .......... .......... .......... .......... .......... 16% 42.2M 5s 2024-08-25T19:47:49.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28050K .......... .......... .......... .......... .......... 16% 20.9M 5s 2024-08-25T19:47:49.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28100K .......... .......... .......... .......... .......... 16% 38.2M 5s 2024-08-25T19:47:49.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28150K .......... .......... .......... .......... .......... 16% 25.5M 5s 2024-08-25T19:47:49.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28200K .......... .......... .......... .......... .......... 16% 42.5M 5s 2024-08-25T19:47:49.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28250K .......... .......... .......... .......... .......... 16% 24.6M 5s 2024-08-25T19:47:49.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28300K .......... .......... .......... .......... .......... 16% 17.5M 5s 2024-08-25T19:47:49.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28350K .......... .......... .......... .......... .......... 16% 22.7M 5s 2024-08-25T19:47:49.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28400K .......... .......... .......... .......... .......... 16% 19.3M 5s 2024-08-25T19:47:49.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28450K .......... .......... .......... .......... .......... 16% 25.9M 5s 2024-08-25T19:47:49.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28500K .......... .......... .......... .......... .......... 16% 35.8M 5s 2024-08-25T19:47:49.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28550K .......... .......... .......... .......... .......... 16% 25.7M 5s 2024-08-25T19:47:49.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28600K .......... .......... .......... .......... .......... 16% 43.6M 5s 2024-08-25T19:47:49.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28650K .......... .......... .......... .......... .......... 17% 22.8M 5s 2024-08-25T19:47:49.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28700K .......... .......... .......... .......... .......... 17% 43.1M 5s 2024-08-25T19:47:49.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28750K .......... .......... .......... .......... .......... 17% 26.6M 5s 2024-08-25T19:47:49.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28800K .......... .......... .......... .......... .......... 17% 24.9M 5s 2024-08-25T19:47:49.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28850K .......... .......... .......... .......... .......... 17% 16.9M 5s 2024-08-25T19:47:49.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28900K .......... .......... .......... .......... .......... 17% 23.4M 5s 2024-08-25T19:47:49.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 28950K .......... .......... .......... .......... .......... 17% 14.8M 5s 2024-08-25T19:47:49.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29000K .......... .......... .......... .......... .......... 17% 43.7M 5s 2024-08-25T19:47:49.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29050K .......... .......... .......... .......... .......... 17% 23.7M 5s 2024-08-25T19:47:49.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29100K .......... .......... .......... .......... .......... 17% 45.9M 5s 2024-08-25T19:47:49.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29150K .......... .......... .......... .......... .......... 17% 14.6M 5s 2024-08-25T19:47:49.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29200K .......... .......... .......... .......... .......... 17% 14.0M 5s 2024-08-25T19:47:49.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29250K .......... .......... .......... .......... .......... 17% 40.7M 5s 2024-08-25T19:47:49.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29300K .......... .......... .......... .......... .......... 17% 23.5M 5s 2024-08-25T19:47:49.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29350K .......... .......... .......... .......... .......... 17% 45.0M 5s 2024-08-25T19:47:49.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29400K .......... .......... .......... .......... .......... 17% 23.0M 5s 2024-08-25T19:47:49.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29450K .......... .......... .......... .......... .......... 17% 22.6M 5s 2024-08-25T19:47:49.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29500K .......... .......... .......... .......... .......... 17% 48.8M 5s 2024-08-25T19:47:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29550K .......... .......... .......... .......... .......... 17% 23.0M 5s 2024-08-25T19:47:49.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29600K .......... .......... .......... .......... .......... 17% 21.8M 5s 2024-08-25T19:47:49.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29650K .......... .......... .......... .......... .......... 17% 47.1M 5s 2024-08-25T19:47:49.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29700K .......... .......... .......... .......... .......... 17% 22.3M 5s 2024-08-25T19:47:49.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29750K .......... .......... .......... .......... .......... 17% 35.8M 5s 2024-08-25T19:47:49.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29800K .......... .......... .......... .......... .......... 17% 28.1M 5s 2024-08-25T19:47:49.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29850K .......... .......... .......... .......... .......... 17% 23.9M 5s 2024-08-25T19:47:49.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29900K .......... .......... .......... .......... .......... 17% 22.6M 5s 2024-08-25T19:47:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 29950K .......... .......... .......... .......... .......... 17% 22.1M 5s 2024-08-25T19:47:49.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30000K .......... .......... .......... .......... .......... 17% 48.0M 5s 2024-08-25T19:47:49.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30050K .......... .......... .......... .......... .......... 17% 23.5M 5s 2024-08-25T19:47:49.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30100K .......... .......... .......... .......... .......... 17% 24.1M 5s 2024-08-25T19:47:49.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30150K .......... .......... .......... .......... .......... 17% 45.0M 5s 2024-08-25T19:47:49.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30200K .......... .......... .......... .......... .......... 17% 22.5M 5s 2024-08-25T19:47:49.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30250K .......... .......... .......... .......... .......... 17% 45.2M 5s 2024-08-25T19:47:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30300K .......... .......... .......... .......... .......... 18% 25.2M 5s 2024-08-25T19:47:49.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30350K .......... .......... .......... .......... .......... 18% 24.5M 5s 2024-08-25T19:47:49.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30400K .......... .......... .......... .......... .......... 18% 35.0M 5s 2024-08-25T19:47:49.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30450K .......... .......... .......... .......... .......... 18% 27.2M 5s 2024-08-25T19:47:49.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30500K .......... .......... .......... .......... .......... 18% 43.6M 5s 2024-08-25T19:47:49.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30550K .......... .......... .......... .......... .......... 18% 23.7M 5s 2024-08-25T19:47:49.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30600K .......... .......... .......... .......... .......... 18% 24.0M 5s 2024-08-25T19:47:49.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30650K .......... .......... .......... .......... .......... 18% 42.6M 5s 2024-08-25T19:47:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30700K .......... .......... .......... .......... .......... 18% 22.8M 5s 2024-08-25T19:47:49.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30750K .......... .......... .......... .......... .......... 18% 21.6M 5s 2024-08-25T19:47:49.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30800K .......... .......... .......... .......... .......... 18% 44.7M 5s 2024-08-25T19:47:49.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30850K .......... .......... .......... .......... .......... 18% 24.4M 5s 2024-08-25T19:47:49.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30900K .......... .......... .......... .......... .......... 18% 30.6M 5s 2024-08-25T19:47:49.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 30950K .......... .......... .......... .......... .......... 18% 22.3M 5s 2024-08-25T19:47:49.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31000K .......... .......... .......... .......... .......... 18% 42.2M 5s 2024-08-25T19:47:49.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31050K .......... .......... .......... .......... .......... 18% 25.0M 5s 2024-08-25T19:47:49.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31100K .......... .......... .......... .......... .......... 18% 29.7M 5s 2024-08-25T19:47:49.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31150K .......... .......... .......... .......... .......... 18% 27.7M 5s 2024-08-25T19:47:49.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31200K .......... .......... .......... .......... .......... 18% 13.4M 5s 2024-08-25T19:47:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31250K .......... .......... .......... .......... .......... 18% 36.4M 5s 2024-08-25T19:47:49.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31300K .......... .......... .......... .......... .......... 18% 46.3M 5s 2024-08-25T19:47:49.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31350K .......... .......... .......... .......... .......... 18% 19.0M 5s 2024-08-25T19:47:49.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31400K .......... .......... .......... .......... .......... 18% 45.7M 5s 2024-08-25T19:47:49.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31450K .......... .......... .......... .......... .......... 18% 24.6M 5s 2024-08-25T19:47:49.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31500K .......... .......... .......... .......... .......... 18% 44.2M 5s 2024-08-25T19:47:49.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31550K .......... .......... .......... .......... .......... 18% 23.4M 5s 2024-08-25T19:47:49.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31600K .......... .......... .......... .......... .......... 18% 18.5M 5s 2024-08-25T19:47:49.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31650K .......... .......... .......... .......... .......... 18% 44.6M 5s 2024-08-25T19:47:49.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31700K .......... .......... .......... .......... .......... 18% 24.2M 5s 2024-08-25T19:47:49.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31750K .......... .......... .......... .......... .......... 18% 20.3M 5s 2024-08-25T19:47:49.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31800K .......... .......... .......... .......... .......... 18% 39.7M 5s 2024-08-25T19:47:49.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31850K .......... .......... .......... .......... .......... 18% 29.9M 5s 2024-08-25T19:47:49.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31900K .......... .......... .......... .......... .......... 18% 30.5M 5s 2024-08-25T19:47:49.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 31950K .......... .......... .......... .......... .......... 18% 22.6M 5s 2024-08-25T19:47:49.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32000K .......... .......... .......... .......... .......... 19% 18.5M 5s 2024-08-25T19:47:49.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32050K .......... .......... .......... .......... .......... 19% 49.1M 5s 2024-08-25T19:47:49.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32100K .......... .......... .......... .......... .......... 19% 22.2M 5s 2024-08-25T19:47:49.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32150K .......... .......... .......... .......... .......... 19% 39.2M 5s 2024-08-25T19:47:49.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32200K .......... .......... .......... .......... .......... 19% 22.2M 5s 2024-08-25T19:47:49.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32250K .......... .......... .......... .......... .......... 19% 47.6M 5s 2024-08-25T19:47:49.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32300K .......... .......... .......... .......... .......... 19% 25.2M 5s 2024-08-25T19:47:49.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32350K .......... .......... .......... .......... .......... 19% 18.8M 5s 2024-08-25T19:47:49.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32400K .......... .......... .......... .......... .......... 19% 24.6M 5s 2024-08-25T19:47:49.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32450K .......... .......... .......... .......... .......... 19% 37.0M 5s 2024-08-25T19:47:49.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32500K .......... .......... .......... .......... .......... 19% 24.9M 5s 2024-08-25T19:47:49.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32550K .......... .......... .......... .......... .......... 19% 29.2M 5s 2024-08-25T19:47:49.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32600K .......... .......... .......... .......... .......... 19% 25.7M 5s 2024-08-25T19:47:49.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32650K .......... .......... .......... .......... .......... 19% 42.3M 5s 2024-08-25T19:47:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32700K .......... .......... .......... .......... .......... 19% 22.3M 5s 2024-08-25T19:47:49.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32750K .......... .......... .......... .......... .......... 19% 25.5M 5s 2024-08-25T19:47:49.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32800K .......... .......... .......... .......... .......... 19% 47.2M 5s 2024-08-25T19:47:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32850K .......... .......... .......... .......... .......... 19% 24.6M 5s 2024-08-25T19:47:49.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32900K .......... .......... .......... .......... .......... 19% 25.6M 5s 2024-08-25T19:47:49.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 32950K .......... .......... .......... .......... .......... 19% 29.6M 5s 2024-08-25T19:47:49.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33000K .......... .......... .......... .......... .......... 19% 15.1M 5s 2024-08-25T19:47:49.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33050K .......... .......... .......... .......... .......... 19% 46.5M 5s 2024-08-25T19:47:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33100K .......... .......... .......... .......... .......... 19% 24.2M 5s 2024-08-25T19:47:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33150K .......... .......... .......... .......... .......... 19% 23.0M 5s 2024-08-25T19:47:49.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33200K .......... .......... .......... .......... .......... 19% 24.0M 5s 2024-08-25T19:47:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33250K .......... .......... .......... .......... .......... 19% 28.2M 5s 2024-08-25T19:47:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33300K .......... .......... .......... .......... .......... 19% 34.5M 5s 2024-08-25T19:47:49.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33350K .......... .......... .......... .......... .......... 19% 24.0M 5s 2024-08-25T19:47:49.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33400K .......... .......... .......... .......... .......... 19% 23.7M 5s 2024-08-25T19:47:49.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33450K .......... .......... .......... .......... .......... 19% 44.4M 5s 2024-08-25T19:47:49.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33500K .......... .......... .......... .......... .......... 19% 12.4M 5s 2024-08-25T19:47:49.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33550K .......... .......... .......... .......... .......... 19% 18.0M 5s 2024-08-25T19:47:49.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33600K .......... .......... .......... .......... .......... 19% 41.8M 5s 2024-08-25T19:47:49.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33650K .......... .......... .......... .......... .......... 19% 21.1M 5s 2024-08-25T19:47:49.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33700K .......... .......... .......... .......... .......... 20% 48.0M 5s 2024-08-25T19:47:49.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33750K .......... .......... .......... .......... .......... 20% 24.2M 5s 2024-08-25T19:47:49.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33800K .......... .......... .......... .......... .......... 20% 42.3M 5s 2024-08-25T19:47:49.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33850K .......... .......... .......... .......... .......... 20% 23.6M 5s 2024-08-25T19:47:49.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33900K .......... .......... .......... .......... .......... 20% 45.2M 5s 2024-08-25T19:47:49.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 33950K .......... .......... .......... .......... .......... 20% 23.1M 5s 2024-08-25T19:47:49.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34000K .......... .......... .......... .......... .......... 20% 21.6M 5s 2024-08-25T19:47:49.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34050K .......... .......... .......... .......... .......... 20% 25.1M 5s 2024-08-25T19:47:49.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34100K .......... .......... .......... .......... .......... 20% 49.9M 5s 2024-08-25T19:47:49.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34150K .......... .......... .......... .......... .......... 20% 23.4M 5s 2024-08-25T19:47:49.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34200K .......... .......... .......... .......... .......... 20% 46.1M 5s 2024-08-25T19:47:49.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34250K .......... .......... .......... .......... .......... 20% 20.8M 5s 2024-08-25T19:47:49.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34300K .......... .......... .......... .......... .......... 20% 57.7M 5s 2024-08-25T19:47:49.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34350K .......... .......... .......... .......... .......... 20% 24.0M 5s 2024-08-25T19:47:49.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34400K .......... .......... .......... .......... .......... 20% 19.0M 5s 2024-08-25T19:47:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34450K .......... .......... .......... .......... .......... 20% 16.3M 5s 2024-08-25T19:47:49.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34500K .......... .......... .......... .......... .......... 20% 26.2M 5s 2024-08-25T19:47:49.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34550K .......... .......... .......... .......... .......... 20% 22.0M 5s 2024-08-25T19:47:49.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34600K .......... .......... .......... .......... .......... 20% 45.7M 5s 2024-08-25T19:47:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34650K .......... .......... .......... .......... .......... 20% 25.9M 5s 2024-08-25T19:47:49.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34700K .......... .......... .......... .......... .......... 20% 41.1M 5s 2024-08-25T19:47:49.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34750K .......... .......... .......... .......... .......... 20% 20.9M 5s 2024-08-25T19:47:49.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34800K .......... .......... .......... .......... .......... 20% 23.0M 5s 2024-08-25T19:47:49.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34850K .......... .......... .......... .......... .......... 20% 46.9M 5s 2024-08-25T19:47:49.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34900K .......... .......... .......... .......... .......... 20% 21.6M 5s 2024-08-25T19:47:49.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 34950K .......... .......... .......... .......... .......... 20% 31.9M 5s 2024-08-25T19:47:49.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35000K .......... .......... .......... .......... .......... 20% 22.7M 5s 2024-08-25T19:47:49.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35050K .......... .......... .......... .......... .......... 20% 37.4M 5s 2024-08-25T19:47:49.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35100K .......... .......... .......... .......... .......... 20% 23.7M 5s 2024-08-25T19:47:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35150K .......... .......... .......... .......... .......... 20% 15.9M 5s 2024-08-25T19:47:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35200K .......... .......... .......... .......... .......... 20% 42.8M 5s 2024-08-25T19:47:49.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35250K .......... .......... .......... .......... .......... 20% 26.7M 5s 2024-08-25T19:47:49.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35300K .......... .......... .......... .......... .......... 20% 25.3M 5s 2024-08-25T19:47:49.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35350K .......... .......... .......... .......... .......... 21% 39.8M 5s 2024-08-25T19:47:49.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35400K .......... .......... .......... .......... .......... 21% 36.2M 5s 2024-08-25T19:47:49.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35450K .......... .......... .......... .......... .......... 21% 15.8M 5s 2024-08-25T19:47:49.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35500K .......... .......... .......... .......... .......... 21% 49.7M 5s 2024-08-25T19:47:49.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35550K .......... .......... .......... .......... .......... 21% 17.4M 5s 2024-08-25T19:47:49.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35600K .......... .......... .......... .......... .......... 21% 43.6M 5s 2024-08-25T19:47:49.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35650K .......... .......... .......... .......... .......... 21% 19.7M 5s 2024-08-25T19:47:49.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35700K .......... .......... .......... .......... .......... 21% 23.9M 5s 2024-08-25T19:47:49.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35750K .......... .......... .......... .......... .......... 21% 41.4M 5s 2024-08-25T19:47:49.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35800K .......... .......... .......... .......... .......... 21% 15.7M 5s 2024-08-25T19:47:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35850K .......... .......... .......... .......... .......... 21% 41.1M 5s 2024-08-25T19:47:49.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35900K .......... .......... .......... .......... .......... 21% 21.4M 5s 2024-08-25T19:47:49.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 35950K .......... .......... .......... .......... .......... 21% 21.7M 5s 2024-08-25T19:47:49.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36000K .......... .......... .......... .......... .......... 21% 41.8M 5s 2024-08-25T19:47:49.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36050K .......... .......... .......... .......... .......... 21% 24.1M 5s 2024-08-25T19:47:49.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36100K .......... .......... .......... .......... .......... 21% 35.4M 5s 2024-08-25T19:47:49.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36150K .......... .......... .......... .......... .......... 21% 19.7M 5s 2024-08-25T19:47:49.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36200K .......... .......... .......... .......... .......... 21% 54.0M 5s 2024-08-25T19:47:49.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36250K .......... .......... .......... .......... .......... 21% 24.1M 5s 2024-08-25T19:47:49.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36300K .......... .......... .......... .......... .......... 21% 17.9M 5s 2024-08-25T19:47:49.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36350K .......... .......... .......... .......... .......... 21% 20.3M 5s 2024-08-25T19:47:49.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36400K .......... .......... .......... .......... .......... 21% 28.0M 5s 2024-08-25T19:47:49.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36450K .......... .......... .......... .......... .......... 21% 24.7M 5s 2024-08-25T19:47:49.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36500K .......... .......... .......... .......... .......... 21% 42.5M 5s 2024-08-25T19:47:49.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36550K .......... .......... .......... .......... .......... 21% 17.3M 5s 2024-08-25T19:47:49.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36600K .......... .......... .......... .......... .......... 21% 43.6M 5s 2024-08-25T19:47:49.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36650K .......... .......... .......... .......... .......... 21% 18.6M 5s 2024-08-25T19:47:49.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36700K .......... .......... .......... .......... .......... 21% 46.2M 5s 2024-08-25T19:47:49.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36750K .......... .......... .......... .......... .......... 21% 20.9M 5s 2024-08-25T19:47:49.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36800K .......... .......... .......... .......... .......... 21% 23.2M 5s 2024-08-25T19:47:49.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36850K .......... .......... .......... .......... .......... 21% 22.7M 5s 2024-08-25T19:47:49.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36900K .......... .......... .......... .......... .......... 21% 17.3M 5s 2024-08-25T19:47:49.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 36950K .......... .......... .......... .......... .......... 21% 26.7M 5s 2024-08-25T19:47:49.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37000K .......... .......... .......... .......... .......... 21% 36.9M 5s 2024-08-25T19:47:49.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37050K .......... .......... .......... .......... .......... 22% 15.9M 5s 2024-08-25T19:47:49.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37100K .......... .......... .......... .......... .......... 22% 41.6M 5s 2024-08-25T19:47:49.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37150K .......... .......... .......... .......... .......... 22% 18.3M 5s 2024-08-25T19:47:49.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37200K .......... .......... .......... .......... .......... 22% 25.5M 5s 2024-08-25T19:47:49.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37250K .......... .......... .......... .......... .......... 22% 44.5M 5s 2024-08-25T19:47:49.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37300K .......... .......... .......... .......... .......... 22% 22.4M 5s 2024-08-25T19:47:49.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37350K .......... .......... .......... .......... .......... 22% 40.2M 5s 2024-08-25T19:47:49.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37400K .......... .......... .......... .......... .......... 22% 27.7M 5s 2024-08-25T19:47:49.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37450K .......... .......... .......... .......... .......... 22% 20.0M 5s 2024-08-25T19:47:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37500K .......... .......... .......... .......... .......... 22% 48.3M 5s 2024-08-25T19:47:49.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37550K .......... .......... .......... .......... .......... 22% 24.1M 5s 2024-08-25T19:47:49.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37600K .......... .......... .......... .......... .......... 22% 23.2M 5s 2024-08-25T19:47:49.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37650K .......... .......... .......... .......... .......... 22% 40.4M 5s 2024-08-25T19:47:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37700K .......... .......... .......... .......... .......... 22% 25.1M 5s 2024-08-25T19:47:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37750K .......... .......... .......... .......... .......... 22% 45.0M 5s 2024-08-25T19:47:49.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37800K .......... .......... .......... .......... .......... 22% 23.7M 5s 2024-08-25T19:47:49.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37850K .......... .......... .......... .......... .......... 22% 37.9M 5s 2024-08-25T19:47:49.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37900K .......... .......... .......... .......... .......... 22% 23.8M 5s 2024-08-25T19:47:49.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 37950K .......... .......... .......... .......... .......... 22% 18.5M 5s 2024-08-25T19:47:49.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38000K .......... .......... .......... .......... .......... 22% 23.9M 5s 2024-08-25T19:47:49.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38050K .......... .......... .......... .......... .......... 22% 52.2M 5s 2024-08-25T19:47:49.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38100K .......... .......... .......... .......... .......... 22% 22.5M 5s 2024-08-25T19:47:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38150K .......... .......... .......... .......... .......... 22% 46.4M 5s 2024-08-25T19:47:49.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38200K .......... .......... .......... .......... .......... 22% 24.8M 5s 2024-08-25T19:47:49.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38250K .......... .......... .......... .......... .......... 22% 30.0M 5s 2024-08-25T19:47:49.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38300K .......... .......... .......... .......... .......... 22% 26.0M 5s 2024-08-25T19:47:49.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38350K .......... .......... .......... .......... .......... 22% 26.2M 5s 2024-08-25T19:47:49.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38400K .......... .......... .......... .......... .......... 22% 41.5M 5s 2024-08-25T19:47:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38450K .......... .......... .......... .......... .......... 22% 23.7M 5s 2024-08-25T19:47:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38500K .......... .......... .......... .......... .......... 22% 38.8M 5s 2024-08-25T19:47:49.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38550K .......... .......... .......... .......... .......... 22% 22.5M 5s 2024-08-25T19:47:49.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38600K .......... .......... .......... .......... .......... 22% 10.7M 5s 2024-08-25T19:47:49.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38650K .......... .......... .......... .......... .......... 22% 87.8M 5s 2024-08-25T19:47:49.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38700K .......... .......... .......... .......... .......... 22% 192M 5s 2024-08-25T19:47:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38750K .......... .......... .......... .......... .......... 23% 23.9M 5s 2024-08-25T19:47:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38800K .......... .......... .......... .......... .......... 23% 40.8M 5s 2024-08-25T19:47:49.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38850K .......... .......... .......... .......... .......... 23% 23.3M 5s 2024-08-25T19:47:49.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38900K .......... .......... .......... .......... .......... 23% 44.6M 5s 2024-08-25T19:47:49.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 38950K .......... .......... .......... .......... .......... 23% 23.5M 5s 2024-08-25T19:47:49.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39000K .......... .......... .......... .......... .......... 23% 40.8M 5s 2024-08-25T19:47:49.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39050K .......... .......... .......... .......... .......... 23% 23.9M 5s 2024-08-25T19:47:49.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39100K .......... .......... .......... .......... .......... 23% 21.3M 5s 2024-08-25T19:47:49.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39150K .......... .......... .......... .......... .......... 23% 25.8M 5s 2024-08-25T19:47:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39200K .......... .......... .......... .......... .......... 23% 40.9M 5s 2024-08-25T19:47:49.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39250K .......... .......... .......... .......... .......... 23% 23.6M 5s 2024-08-25T19:47:49.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39300K .......... .......... .......... .......... .......... 23% 41.6M 5s 2024-08-25T19:47:49.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39350K .......... .......... .......... .......... .......... 23% 21.8M 5s 2024-08-25T19:47:49.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39400K .......... .......... .......... .......... .......... 23% 53.8M 5s 2024-08-25T19:47:49.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39450K .......... .......... .......... .......... .......... 23% 24.6M 5s 2024-08-25T19:47:49.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39500K .......... .......... .......... .......... .......... 23% 37.6M 5s 2024-08-25T19:47:49.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39550K .......... .......... .......... .......... .......... 23% 24.1M 5s 2024-08-25T19:47:49.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39600K .......... .......... .......... .......... .......... 23% 20.4M 5s 2024-08-25T19:47:49.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39650K .......... .......... .......... .......... .......... 23% 27.4M 5s 2024-08-25T19:47:49.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39700K .......... .......... .......... .......... .......... 23% 43.8M 5s 2024-08-25T19:47:49.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39750K .......... .......... .......... .......... .......... 23% 25.2M 5s 2024-08-25T19:47:49.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39800K .......... .......... .......... .......... .......... 23% 45.9M 5s 2024-08-25T19:47:49.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39850K .......... .......... .......... .......... .......... 23% 22.4M 5s 2024-08-25T19:47:49.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39900K .......... .......... .......... .......... .......... 23% 45.4M 5s 2024-08-25T19:47:49.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 39950K .......... .......... .......... .......... .......... 23% 22.4M 5s 2024-08-25T19:47:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40000K .......... .......... .......... .......... .......... 23% 28.4M 5s 2024-08-25T19:47:49.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40050K .......... .......... .......... .......... .......... 23% 33.5M 5s 2024-08-25T19:47:49.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40100K .......... .......... .......... .......... .......... 23% 36.7M 5s 2024-08-25T19:47:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40150K .......... .......... .......... .......... .......... 23% 27.6M 5s 2024-08-25T19:47:49.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40200K .......... .......... .......... .......... .......... 23% 22.8M 5s 2024-08-25T19:47:49.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40250K .......... .......... .......... .......... .......... 23% 36.9M 5s 2024-08-25T19:47:49.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40300K .......... .......... .......... .......... .......... 23% 19.4M 5s 2024-08-25T19:47:49.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40350K .......... .......... .......... .......... .......... 23% 28.5M 5s 2024-08-25T19:47:49.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40400K .......... .......... .......... .......... .......... 23% 25.1M 5s 2024-08-25T19:47:49.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40450K .......... .......... .......... .......... .......... 24% 44.2M 5s 2024-08-25T19:47:49.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40500K .......... .......... .......... .......... .......... 24% 23.9M 5s 2024-08-25T19:47:49.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40550K .......... .......... .......... .......... .......... 24% 46.8M 5s 2024-08-25T19:47:49.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40600K .......... .......... .......... .......... .......... 24% 19.8M 5s 2024-08-25T19:47:49.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40650K .......... .......... .......... .......... .......... 24% 27.4M 5s 2024-08-25T19:47:49.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40700K .......... .......... .......... .......... .......... 24% 12.7M 5s 2024-08-25T19:47:49.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40750K .......... .......... .......... .......... .......... 24% 23.5M 5s 2024-08-25T19:47:49.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40800K .......... .......... .......... .......... .......... 24% 28.8M 5s 2024-08-25T19:47:49.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40850K .......... .......... .......... .......... .......... 24% 22.5M 5s 2024-08-25T19:47:50.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40900K .......... .......... .......... .......... .......... 24% 24.2M 5s 2024-08-25T19:47:50.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 40950K .......... .......... .......... .......... .......... 24% 53.3M 5s 2024-08-25T19:47:50.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41000K .......... .......... .......... .......... .......... 24% 24.2M 5s 2024-08-25T19:47:50.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41050K .......... .......... .......... .......... .......... 24% 31.5M 5s 2024-08-25T19:47:50.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41100K .......... .......... .......... .......... .......... 24% 19.4M 5s 2024-08-25T19:47:50.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41150K .......... .......... .......... .......... .......... 24% 22.1M 5s 2024-08-25T19:47:50.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41200K .......... .......... .......... .......... .......... 24% 25.8M 5s 2024-08-25T19:47:50.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41250K .......... .......... .......... .......... .......... 24% 26.0M 5s 2024-08-25T19:47:50.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41300K .......... .......... .......... .......... .......... 24% 38.7M 5s 2024-08-25T19:47:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41350K .......... .......... .......... .......... .......... 24% 23.7M 5s 2024-08-25T19:47:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41400K .......... .......... .......... .......... .......... 24% 22.0M 5s 2024-08-25T19:47:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41450K .......... .......... .......... .......... .......... 24% 60.4M 5s 2024-08-25T19:47:50.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41500K .......... .......... .......... .......... .......... 24% 14.8M 5s 2024-08-25T19:47:50.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41550K .......... .......... .......... .......... .......... 24% 23.8M 5s 2024-08-25T19:47:50.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41600K .......... .......... .......... .......... .......... 24% 34.1M 5s 2024-08-25T19:47:50.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41650K .......... .......... .......... .......... .......... 24% 18.5M 5s 2024-08-25T19:47:50.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41700K .......... .......... .......... .......... .......... 24% 50.7M 5s 2024-08-25T19:47:50.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41750K .......... .......... .......... .......... .......... 24% 22.6M 5s 2024-08-25T19:47:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41800K .......... .......... .......... .......... .......... 24% 38.1M 5s 2024-08-25T19:47:50.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41850K .......... .......... .......... .......... .......... 24% 25.5M 5s 2024-08-25T19:47:50.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41900K .......... .......... .......... .......... .......... 24% 18.2M 5s 2024-08-25T19:47:50.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 41950K .......... .......... .......... .......... .......... 24% 16.4M 5s 2024-08-25T19:47:50.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42000K .......... .......... .......... .......... .......... 24% 51.2M 5s 2024-08-25T19:47:50.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42050K .......... .......... .......... .......... .......... 24% 22.0M 5s 2024-08-25T19:47:50.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42100K .......... .......... .......... .......... .......... 25% 39.1M 5s 2024-08-25T19:47:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42150K .......... .......... .......... .......... .......... 25% 24.3M 5s 2024-08-25T19:47:50.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42200K .......... .......... .......... .......... .......... 25% 44.9M 5s 2024-08-25T19:47:50.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42250K .......... .......... .......... .......... .......... 25% 25.0M 5s 2024-08-25T19:47:50.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42300K .......... .......... .......... .......... .......... 25% 26.2M 5s 2024-08-25T19:47:50.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42350K .......... .......... .......... .......... .......... 25% 27.7M 5s 2024-08-25T19:47:50.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42400K .......... .......... .......... .......... .......... 25% 19.6M 5s 2024-08-25T19:47:50.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42450K .......... .......... .......... .......... .......... 25% 48.5M 5s 2024-08-25T19:47:50.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42500K .......... .......... .......... .......... .......... 25% 19.4M 5s 2024-08-25T19:47:50.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42550K .......... .......... .......... .......... .......... 25% 19.0M 5s 2024-08-25T19:47:50.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42600K .......... .......... .......... .......... .......... 25% 61.5M 5s 2024-08-25T19:47:50.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42650K .......... .......... .......... .......... .......... 25% 20.5M 5s 2024-08-25T19:47:50.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42700K .......... .......... .......... .......... .......... 25% 36.5M 5s 2024-08-25T19:47:50.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42750K .......... .......... .......... .......... .......... 25% 18.4M 5s 2024-08-25T19:47:50.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42800K .......... .......... .......... .......... .......... 25% 27.5M 5s 2024-08-25T19:47:50.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42850K .......... .......... .......... .......... .......... 25% 39.9M 5s 2024-08-25T19:47:50.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42900K .......... .......... .......... .......... .......... 25% 24.8M 5s 2024-08-25T19:47:50.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 42950K .......... .......... .......... .......... .......... 25% 36.5M 5s 2024-08-25T19:47:50.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43000K .......... .......... .......... .......... .......... 25% 24.5M 5s 2024-08-25T19:47:50.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43050K .......... .......... .......... .......... .......... 25% 21.5M 5s 2024-08-25T19:47:50.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43100K .......... .......... .......... .......... .......... 25% 54.4M 5s 2024-08-25T19:47:50.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43150K .......... .......... .......... .......... .......... 25% 21.6M 5s 2024-08-25T19:47:50.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43200K .......... .......... .......... .......... .......... 25% 21.8M 5s 2024-08-25T19:47:50.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43250K .......... .......... .......... .......... .......... 25% 41.7M 5s 2024-08-25T19:47:50.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43300K .......... .......... .......... .......... .......... 25% 23.9M 5s 2024-08-25T19:47:50.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43350K .......... .......... .......... .......... .......... 25% 40.2M 5s 2024-08-25T19:47:50.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43400K .......... .......... .......... .......... .......... 25% 26.4M 5s 2024-08-25T19:47:50.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43450K .......... .......... .......... .......... .......... 25% 38.0M 5s 2024-08-25T19:47:50.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43500K .......... .......... .......... .......... .......... 25% 24.3M 5s 2024-08-25T19:47:50.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43550K .......... .......... .......... .......... .......... 25% 20.1M 5s 2024-08-25T19:47:50.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43600K .......... .......... .......... .......... .......... 25% 22.2M 5s 2024-08-25T19:47:50.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43650K .......... .......... .......... .......... .......... 25% 46.5M 5s 2024-08-25T19:47:50.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43700K .......... .......... .......... .......... .......... 25% 24.7M 5s 2024-08-25T19:47:50.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43750K .......... .......... .......... .......... .......... 25% 39.4M 5s 2024-08-25T19:47:50.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43800K .......... .......... .......... .......... .......... 26% 26.0M 5s 2024-08-25T19:47:50.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43850K .......... .......... .......... .......... .......... 26% 39.1M 5s 2024-08-25T19:47:50.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43900K .......... .......... .......... .......... .......... 26% 25.4M 5s 2024-08-25T19:47:50.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 43950K .......... .......... .......... .......... .......... 26% 24.5M 5s 2024-08-25T19:47:50.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44000K .......... .......... .......... .......... .......... 26% 43.8M 5s 2024-08-25T19:47:50.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44050K .......... .......... .......... .......... .......... 26% 22.5M 5s 2024-08-25T19:47:50.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44100K .......... .......... .......... .......... .......... 26% 37.9M 5s 2024-08-25T19:47:50.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44150K .......... .......... .......... .......... .......... 26% 26.9M 5s 2024-08-25T19:47:50.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44200K .......... .......... .......... .......... .......... 26% 24.7M 5s 2024-08-25T19:47:50.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44250K .......... .......... .......... .......... .......... 26% 46.9M 5s 2024-08-25T19:47:50.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44300K .......... .......... .......... .......... .......... 26% 23.3M 5s 2024-08-25T19:47:50.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44350K .......... .......... .......... .......... .......... 26% 20.9M 5s 2024-08-25T19:47:50.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44400K .......... .......... .......... .......... .......... 26% 42.5M 5s 2024-08-25T19:47:50.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44450K .......... .......... .......... .......... .......... 26% 24.6M 5s 2024-08-25T19:47:50.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44500K .......... .......... .......... .......... .......... 26% 43.6M 5s 2024-08-25T19:47:50.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44550K .......... .......... .......... .......... .......... 26% 23.7M 5s 2024-08-25T19:47:50.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44600K .......... .......... .......... .......... .......... 26% 42.0M 5s 2024-08-25T19:47:50.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44650K .......... .......... .......... .......... .......... 26% 25.5M 5s 2024-08-25T19:47:50.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44700K .......... .......... .......... .......... .......... 26% 42.0M 5s 2024-08-25T19:47:50.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44750K .......... .......... .......... .......... .......... 26% 11.1M 5s 2024-08-25T19:47:50.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44800K .......... .......... .......... .......... .......... 26% 45.6M 5s 2024-08-25T19:47:50.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44850K .......... .......... .......... .......... .......... 26% 31.2M 5s 2024-08-25T19:47:50.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44900K .......... .......... .......... .......... .......... 26% 27.6M 5s 2024-08-25T19:47:50.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 44950K .......... .......... .......... .......... .......... 26% 24.4M 5s 2024-08-25T19:47:50.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45000K .......... .......... .......... .......... .......... 26% 28.9M 5s 2024-08-25T19:47:50.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45050K .......... .......... .......... .......... .......... 26% 22.0M 5s 2024-08-25T19:47:50.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45100K .......... .......... .......... .......... .......... 26% 48.0M 5s 2024-08-25T19:47:50.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45150K .......... .......... .......... .......... .......... 26% 23.6M 5s 2024-08-25T19:47:50.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45200K .......... .......... .......... .......... .......... 26% 24.9M 5s 2024-08-25T19:47:50.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45250K .......... .......... .......... .......... .......... 26% 21.9M 5s 2024-08-25T19:47:50.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45300K .......... .......... .......... .......... .......... 26% 23.9M 5s 2024-08-25T19:47:50.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45350K .......... .......... .......... .......... .......... 26% 28.1M 5s 2024-08-25T19:47:50.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45400K .......... .......... .......... .......... .......... 26% 29.9M 5s 2024-08-25T19:47:50.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45450K .......... .......... .......... .......... .......... 26% 22.8M 5s 2024-08-25T19:47:50.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45500K .......... .......... .......... .......... .......... 27% 56.7M 5s 2024-08-25T19:47:50.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45550K .......... .......... .......... .......... .......... 27% 23.8M 5s 2024-08-25T19:47:50.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45600K .......... .......... .......... .......... .......... 27% 21.3M 5s 2024-08-25T19:47:50.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45650K .......... .......... .......... .......... .......... 27% 42.7M 5s 2024-08-25T19:47:50.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45700K .......... .......... .......... .......... .......... 27% 22.3M 5s 2024-08-25T19:47:50.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45750K .......... .......... .......... .......... .......... 27% 16.5M 5s 2024-08-25T19:47:50.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45800K .......... .......... .......... .......... .......... 27% 23.6M 5s 2024-08-25T19:47:50.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45850K .......... .......... .......... .......... .......... 27% 40.3M 5s 2024-08-25T19:47:50.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45900K .......... .......... .......... .......... .......... 27% 13.8M 5s 2024-08-25T19:47:50.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 45950K .......... .......... .......... .......... .......... 27% 21.0M 5s 2024-08-25T19:47:50.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46000K .......... .......... .......... .......... .......... 27% 13.9M 5s 2024-08-25T19:47:50.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46050K .......... .......... .......... .......... .......... 27% 53.8M 5s 2024-08-25T19:47:50.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46100K .......... .......... .......... .......... .......... 27% 24.2M 5s 2024-08-25T19:47:50.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46150K .......... .......... .......... .......... .......... 27% 40.2M 5s 2024-08-25T19:47:50.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46200K .......... .......... .......... .......... .......... 27% 18.2M 5s 2024-08-25T19:47:50.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46250K .......... .......... .......... .......... .......... 27% 43.7M 5s 2024-08-25T19:47:50.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46300K .......... .......... .......... .......... .......... 27% 25.0M 5s 2024-08-25T19:47:50.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46350K .......... .......... .......... .......... .......... 27% 23.2M 5s 2024-08-25T19:47:50.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46400K .......... .......... .......... .......... .......... 27% 31.6M 5s 2024-08-25T19:47:50.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46450K .......... .......... .......... .......... .......... 27% 26.9M 5s 2024-08-25T19:47:50.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46500K .......... .......... .......... .......... .......... 27% 33.1M 5s 2024-08-25T19:47:50.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46550K .......... .......... .......... .......... .......... 27% 32.4M 5s 2024-08-25T19:47:50.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46600K .......... .......... .......... .......... .......... 27% 21.6M 5s 2024-08-25T19:47:50.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46650K .......... .......... .......... .......... .......... 27% 42.8M 5s 2024-08-25T19:47:50.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46700K .......... .......... .......... .......... .......... 27% 23.4M 5s 2024-08-25T19:47:50.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46750K .......... .......... .......... .......... .......... 27% 21.5M 5s 2024-08-25T19:47:50.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46800K .......... .......... .......... .......... .......... 27% 52.3M 5s 2024-08-25T19:47:50.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46850K .......... .......... .......... .......... .......... 27% 23.3M 5s 2024-08-25T19:47:50.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46900K .......... .......... .......... .......... .......... 27% 39.8M 5s 2024-08-25T19:47:50.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 46950K .......... .......... .......... .......... .......... 27% 20.6M 5s 2024-08-25T19:47:50.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47000K .......... .......... .......... .......... .......... 27% 49.6M 5s 2024-08-25T19:47:50.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47050K .......... .......... .......... .......... .......... 27% 21.1M 5s 2024-08-25T19:47:50.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47100K .......... .......... .......... .......... .......... 27% 21.0M 5s 2024-08-25T19:47:50.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47150K .......... .......... .......... .......... .......... 28% 20.5M 5s 2024-08-25T19:47:50.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47200K .......... .......... .......... .......... .......... 28% 44.2M 5s 2024-08-25T19:47:50.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47250K .......... .......... .......... .......... .......... 28% 24.0M 5s 2024-08-25T19:47:50.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47300K .......... .......... .......... .......... .......... 28% 42.6M 5s 2024-08-25T19:47:50.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47350K .......... .......... .......... .......... .......... 28% 25.4M 5s 2024-08-25T19:47:50.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47400K .......... .......... .......... .......... .......... 28% 38.2M 5s 2024-08-25T19:47:50.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47450K .......... .......... .......... .......... .......... 28% 23.9M 5s 2024-08-25T19:47:50.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47500K .......... .......... .......... .......... .......... 28% 32.1M 5s 2024-08-25T19:47:50.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47550K .......... .......... .......... .......... .......... 28% 19.3M 5s 2024-08-25T19:47:50.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47600K .......... .......... .......... .......... .......... 28% 21.9M 5s 2024-08-25T19:47:50.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47650K .......... .......... .......... .......... .......... 28% 34.3M 5s 2024-08-25T19:47:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47700K .......... .......... .......... .......... .......... 28% 24.9M 5s 2024-08-25T19:47:50.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47750K .......... .......... .......... .......... .......... 28% 26.1M 5s 2024-08-25T19:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47800K .......... .......... .......... .......... .......... 28% 40.8M 5s 2024-08-25T19:47:50.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47850K .......... .......... .......... .......... .......... 28% 26.5M 5s 2024-08-25T19:47:50.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47900K .......... .......... .......... .......... .......... 28% 40.3M 5s 2024-08-25T19:47:50.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 47950K .......... .......... .......... .......... .......... 28% 23.9M 5s 2024-08-25T19:47:50.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48000K .......... .......... .......... .......... .......... 28% 21.8M 5s 2024-08-25T19:47:50.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48050K .......... .......... .......... .......... .......... 28% 41.6M 5s 2024-08-25T19:47:50.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48100K .......... .......... .......... .......... .......... 28% 21.4M 5s 2024-08-25T19:47:50.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48150K .......... .......... .......... .......... .......... 28% 23.9M 5s 2024-08-25T19:47:50.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48200K .......... .......... .......... .......... .......... 28% 43.9M 5s 2024-08-25T19:47:50.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48250K .......... .......... .......... .......... .......... 28% 22.8M 5s 2024-08-25T19:47:50.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48300K .......... .......... .......... .......... .......... 28% 38.1M 5s 2024-08-25T19:47:50.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48350K .......... .......... .......... .......... .......... 28% 24.3M 5s 2024-08-25T19:47:50.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48400K .......... .......... .......... .......... .......... 28% 26.8M 5s 2024-08-25T19:47:50.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48450K .......... .......... .......... .......... .......... 28% 45.8M 5s 2024-08-25T19:47:50.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48500K .......... .......... .......... .......... .......... 28% 18.5M 5s 2024-08-25T19:47:50.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48550K .......... .......... .......... .......... .......... 28% 43.5M 5s 2024-08-25T19:47:50.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48600K .......... .......... .......... .......... .......... 28% 24.8M 5s 2024-08-25T19:47:50.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48650K .......... .......... .......... .......... .......... 28% 40.1M 5s 2024-08-25T19:47:50.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48700K .......... .......... .......... .......... .......... 28% 16.0M 5s 2024-08-25T19:47:50.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48750K .......... .......... .......... .......... .......... 28% 22.6M 5s 2024-08-25T19:47:50.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48800K .......... .......... .......... .......... .......... 28% 24.9M 5s 2024-08-25T19:47:50.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48850K .......... .......... .......... .......... .......... 29% 38.7M 5s 2024-08-25T19:47:50.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48900K .......... .......... .......... .......... .......... 29% 20.9M 5s 2024-08-25T19:47:50.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 48950K .......... .......... .......... .......... .......... 29% 23.4M 5s 2024-08-25T19:47:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49000K .......... .......... .......... .......... .......... 29% 23.7M 5s 2024-08-25T19:47:50.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49050K .......... .......... .......... .......... .......... 29% 39.0M 5s 2024-08-25T19:47:50.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49100K .......... .......... .......... .......... .......... 29% 15.5M 5s 2024-08-25T19:47:50.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49150K .......... .......... .......... .......... .......... 29% 19.6M 5s 2024-08-25T19:47:50.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49200K .......... .......... .......... .......... .......... 29% 31.9M 5s 2024-08-25T19:47:50.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49250K .......... .......... .......... .......... .......... 29% 30.7M 5s 2024-08-25T19:47:50.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49300K .......... .......... .......... .......... .......... 29% 20.1M 5s 2024-08-25T19:47:50.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49350K .......... .......... .......... .......... .......... 29% 30.9M 5s 2024-08-25T19:47:50.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49400K .......... .......... .......... .......... .......... 29% 28.4M 5s 2024-08-25T19:47:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49450K .......... .......... .......... .......... .......... 29% 32.1M 5s 2024-08-25T19:47:50.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49500K .......... .......... .......... .......... .......... 29% 18.2M 5s 2024-08-25T19:47:50.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49550K .......... .......... .......... .......... .......... 29% 22.0M 5s 2024-08-25T19:47:50.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49600K .......... .......... .......... .......... .......... 29% 36.4M 5s 2024-08-25T19:47:50.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49650K .......... .......... .......... .......... .......... 29% 31.6M 5s 2024-08-25T19:47:50.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49700K .......... .......... .......... .......... .......... 29% 32.9M 4s 2024-08-25T19:47:50.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49750K .......... .......... .......... .......... .......... 29% 19.9M 4s 2024-08-25T19:47:50.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49800K .......... .......... .......... .......... .......... 29% 24.6M 4s 2024-08-25T19:47:50.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49850K .......... .......... .......... .......... .......... 29% 27.3M 4s 2024-08-25T19:47:50.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49900K .......... .......... .......... .......... .......... 29% 34.2M 4s 2024-08-25T19:47:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 49950K .......... .......... .......... .......... .......... 29% 21.1M 4s 2024-08-25T19:47:50.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50000K .......... .......... .......... .......... .......... 29% 29.4M 4s 2024-08-25T19:47:50.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50050K .......... .......... .......... .......... .......... 29% 17.3M 4s 2024-08-25T19:47:50.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50100K .......... .......... .......... .......... .......... 29% 37.5M 4s 2024-08-25T19:47:50.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50150K .......... .......... .......... .......... .......... 29% 25.8M 4s 2024-08-25T19:47:50.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50200K .......... .......... .......... .......... .......... 29% 32.3M 4s 2024-08-25T19:47:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50250K .......... .......... .......... .......... .......... 29% 15.6M 4s 2024-08-25T19:47:50.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50300K .......... .......... .......... .......... .......... 29% 39.4M 4s 2024-08-25T19:47:50.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50350K .......... .......... .......... .......... .......... 29% 18.4M 4s 2024-08-25T19:47:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50400K .......... .......... .......... .......... .......... 29% 32.0M 4s 2024-08-25T19:47:50.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50450K .......... .......... .......... .......... .......... 29% 22.9M 4s 2024-08-25T19:47:50.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50500K .......... .......... .......... .......... .......... 29% 37.1M 4s 2024-08-25T19:47:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50550K .......... .......... .......... .......... .......... 30% 24.8M 4s 2024-08-25T19:47:50.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50600K .......... .......... .......... .......... .......... 30% 35.5M 4s 2024-08-25T19:47:50.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50650K .......... .......... .......... .......... .......... 30% 15.9M 4s 2024-08-25T19:47:50.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50700K .......... .......... .......... .......... .......... 30% 21.1M 4s 2024-08-25T19:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50750K .......... .......... .......... .......... .......... 30% 29.5M 4s 2024-08-25T19:47:50.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50800K .......... .......... .......... .......... .......... 30% 25.2M 4s 2024-08-25T19:47:50.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50850K .......... .......... .......... .......... .......... 30% 43.2M 4s 2024-08-25T19:47:50.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50900K .......... .......... .......... .......... .......... 30% 25.5M 4s 2024-08-25T19:47:50.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 50950K .......... .......... .......... .......... .......... 30% 38.4M 4s 2024-08-25T19:47:50.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51000K .......... .......... .......... .......... .......... 30% 22.8M 4s 2024-08-25T19:47:50.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51050K .......... .......... .......... .......... .......... 30% 22.9M 4s 2024-08-25T19:47:50.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51100K .......... .......... .......... .......... .......... 30% 45.8M 4s 2024-08-25T19:47:50.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51150K .......... .......... .......... .......... .......... 30% 24.7M 4s 2024-08-25T19:47:50.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51200K .......... .......... .......... .......... .......... 30% 20.7M 4s 2024-08-25T19:47:50.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51250K .......... .......... .......... .......... .......... 30% 36.7M 4s 2024-08-25T19:47:50.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51300K .......... .......... .......... .......... .......... 30% 25.7M 4s 2024-08-25T19:47:50.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51350K .......... .......... .......... .......... .......... 30% 35.9M 4s 2024-08-25T19:47:50.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51400K .......... .......... .......... .......... .......... 30% 29.2M 4s 2024-08-25T19:47:50.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51450K .......... .......... .......... .......... .......... 30% 36.0M 4s 2024-08-25T19:47:50.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51500K .......... .......... .......... .......... .......... 30% 24.3M 4s 2024-08-25T19:47:50.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51550K .......... .......... .......... .......... .......... 30% 21.0M 4s 2024-08-25T19:47:50.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51600K .......... .......... .......... .......... .......... 30% 26.4M 4s 2024-08-25T19:47:50.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51650K .......... .......... .......... .......... .......... 30% 37.9M 4s 2024-08-25T19:47:50.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51700K .......... .......... .......... .......... .......... 30% 25.3M 4s 2024-08-25T19:47:50.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51750K .......... .......... .......... .......... .......... 30% 36.7M 4s 2024-08-25T19:47:50.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51800K .......... .......... .......... .......... .......... 30% 28.2M 4s 2024-08-25T19:47:50.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51850K .......... .......... .......... .......... .......... 30% 47.2M 4s 2024-08-25T19:47:50.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51900K .......... .......... .......... .......... .......... 30% 19.1M 4s 2024-08-25T19:47:50.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 51950K .......... .......... .......... .......... .......... 30% 26.1M 4s 2024-08-25T19:47:50.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52000K .......... .......... .......... .......... .......... 30% 32.5M 4s 2024-08-25T19:47:50.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52050K .......... .......... .......... .......... .......... 30% 28.8M 4s 2024-08-25T19:47:50.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52100K .......... .......... .......... .......... .......... 30% 22.7M 4s 2024-08-25T19:47:50.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52150K .......... .......... .......... .......... .......... 30% 47.5M 4s 2024-08-25T19:47:50.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52200K .......... .......... .......... .......... .......... 30% 24.6M 4s 2024-08-25T19:47:50.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52250K .......... .......... .......... .......... .......... 31% 31.8M 4s 2024-08-25T19:47:50.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52300K .......... .......... .......... .......... .......... 31% 26.4M 4s 2024-08-25T19:47:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52350K .......... .......... .......... .......... .......... 31% 25.2M 4s 2024-08-25T19:47:50.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52400K .......... .......... .......... .......... .......... 31% 45.2M 4s 2024-08-25T19:47:50.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52450K .......... .......... .......... .......... .......... 31% 22.9M 4s 2024-08-25T19:47:50.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52500K .......... .......... .......... .......... .......... 31% 39.2M 4s 2024-08-25T19:47:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52550K .......... .......... .......... .......... .......... 31% 23.9M 4s 2024-08-25T19:47:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52600K .......... .......... .......... .......... .......... 31% 39.3M 4s 2024-08-25T19:47:50.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52650K .......... .......... .......... .......... .......... 31% 22.9M 4s 2024-08-25T19:47:50.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52700K .......... .......... .......... .......... .......... 31% 22.4M 4s 2024-08-25T19:47:50.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52750K .......... .......... .......... .......... .......... 31% 27.1M 4s 2024-08-25T19:47:50.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52800K .......... .......... .......... .......... .......... 31% 39.0M 4s 2024-08-25T19:47:50.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52850K .......... .......... .......... .......... .......... 31% 16.5M 4s 2024-08-25T19:47:50.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52900K .......... .......... .......... .......... .......... 31% 38.1M 4s 2024-08-25T19:47:50.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 52950K .......... .......... .......... .......... .......... 31% 22.9M 4s 2024-08-25T19:47:50.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53000K .......... .......... .......... .......... .......... 31% 27.7M 4s 2024-08-25T19:47:50.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53050K .......... .......... .......... .......... .......... 31% 26.0M 4s 2024-08-25T19:47:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53100K .......... .......... .......... .......... .......... 31% 40.6M 4s 2024-08-25T19:47:50.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53150K .......... .......... .......... .......... .......... 31% 22.6M 4s 2024-08-25T19:47:50.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53200K .......... .......... .......... .......... .......... 31% 24.2M 4s 2024-08-25T19:47:50.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53250K .......... .......... .......... .......... .......... 31% 17.1M 4s 2024-08-25T19:47:50.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53300K .......... .......... .......... .......... .......... 31% 33.5M 4s 2024-08-25T19:47:50.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53350K .......... .......... .......... .......... .......... 31% 24.1M 4s 2024-08-25T19:47:50.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53400K .......... .......... .......... .......... .......... 31% 35.5M 4s 2024-08-25T19:47:50.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53450K .......... .......... .......... .......... .......... 31% 31.7M 4s 2024-08-25T19:47:50.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53500K .......... .......... .......... .......... .......... 31% 13.8M 4s 2024-08-25T19:47:50.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53550K .......... .......... .......... .......... .......... 31% 21.6M 4s 2024-08-25T19:47:50.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53600K .......... .......... .......... .......... .......... 31% 24.3M 4s 2024-08-25T19:47:50.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53650K .......... .......... .......... .......... .......... 31% 44.2M 4s 2024-08-25T19:47:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53700K .......... .......... .......... .......... .......... 31% 23.4M 4s 2024-08-25T19:47:50.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53750K .......... .......... .......... .......... .......... 31% 40.7M 4s 2024-08-25T19:47:50.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53800K .......... .......... .......... .......... .......... 31% 21.5M 4s 2024-08-25T19:47:50.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53850K .......... .......... .......... .......... .......... 31% 25.5M 4s 2024-08-25T19:47:50.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53900K .......... .......... .......... .......... .......... 32% 22.7M 4s 2024-08-25T19:47:50.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 53950K .......... .......... .......... .......... .......... 32% 28.0M 4s 2024-08-25T19:47:50.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54000K .......... .......... .......... .......... .......... 32% 23.5M 4s 2024-08-25T19:47:50.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54050K .......... .......... .......... .......... .......... 32% 42.3M 4s 2024-08-25T19:47:50.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54100K .......... .......... .......... .......... .......... 32% 24.0M 4s 2024-08-25T19:47:50.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54150K .......... .......... .......... .......... .......... 32% 42.6M 4s 2024-08-25T19:47:50.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54200K .......... .......... .......... .......... .......... 32% 12.8M 4s 2024-08-25T19:47:50.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54250K .......... .......... .......... .......... .......... 32% 41.3M 4s 2024-08-25T19:47:50.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54300K .......... .......... .......... .......... .......... 32% 22.3M 4s 2024-08-25T19:47:50.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54350K .......... .......... .......... .......... .......... 32% 24.7M 4s 2024-08-25T19:47:50.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54400K .......... .......... .......... .......... .......... 32% 23.1M 4s 2024-08-25T19:47:50.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54450K .......... .......... .......... .......... .......... 32% 33.1M 4s 2024-08-25T19:47:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54500K .......... .......... .......... .......... .......... 32% 23.2M 4s 2024-08-25T19:47:50.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54550K .......... .......... .......... .......... .......... 32% 39.2M 4s 2024-08-25T19:47:50.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54600K .......... .......... .......... .......... .......... 32% 17.8M 4s 2024-08-25T19:47:50.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54650K .......... .......... .......... .......... .......... 32% 46.8M 4s 2024-08-25T19:47:50.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54700K .......... .......... .......... .......... .......... 32% 23.2M 4s 2024-08-25T19:47:50.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54750K .......... .......... .......... .......... .......... 32% 24.9M 4s 2024-08-25T19:47:50.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54800K .......... .......... .......... .......... .......... 32% 36.7M 4s 2024-08-25T19:47:50.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54850K .......... .......... .......... .......... .......... 32% 22.5M 4s 2024-08-25T19:47:50.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54900K .......... .......... .......... .......... .......... 32% 47.3M 4s 2024-08-25T19:47:50.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 54950K .......... .......... .......... .......... .......... 32% 23.6M 4s 2024-08-25T19:47:50.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55000K .......... .......... .......... .......... .......... 32% 29.8M 4s 2024-08-25T19:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55050K .......... .......... .......... .......... .......... 32% 27.2M 4s 2024-08-25T19:47:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55100K .......... .......... .......... .......... .......... 32% 24.6M 4s 2024-08-25T19:47:50.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55150K .......... .......... .......... .......... .......... 32% 20.7M 4s 2024-08-25T19:47:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55200K .......... .......... .......... .......... .......... 32% 48.4M 4s 2024-08-25T19:47:50.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55250K .......... .......... .......... .......... .......... 32% 23.6M 4s 2024-08-25T19:47:50.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55300K .......... .......... .......... .......... .......... 32% 37.1M 4s 2024-08-25T19:47:50.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55350K .......... .......... .......... .......... .......... 32% 17.2M 4s 2024-08-25T19:47:50.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55400K .......... .......... .......... .......... .......... 32% 49.0M 4s 2024-08-25T19:47:50.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55450K .......... .......... .......... .......... .......... 32% 23.2M 4s 2024-08-25T19:47:50.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55500K .......... .......... .......... .......... .......... 32% 42.0M 4s 2024-08-25T19:47:50.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55550K .......... .......... .......... .......... .......... 32% 14.7M 4s 2024-08-25T19:47:50.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55600K .......... .......... .......... .......... .......... 33% 40.6M 4s 2024-08-25T19:47:50.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55650K .......... .......... .......... .......... .......... 33% 25.4M 4s 2024-08-25T19:47:50.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55700K .......... .......... .......... .......... .......... 33% 40.7M 4s 2024-08-25T19:47:50.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55750K .......... .......... .......... .......... .......... 33% 34.9M 4s 2024-08-25T19:47:50.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55800K .......... .......... .......... .......... .......... 33% 23.2M 4s 2024-08-25T19:47:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55850K .......... .......... .......... .......... .......... 33% 24.6M 4s 2024-08-25T19:47:50.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55900K .......... .......... .......... .......... .......... 33% 45.4M 4s 2024-08-25T19:47:50.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 55950K .......... .......... .......... .......... .......... 33% 21.8M 4s 2024-08-25T19:47:50.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56000K .......... .......... .......... .......... .......... 33% 24.6M 4s 2024-08-25T19:47:50.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56050K .......... .......... .......... .......... .......... 33% 38.5M 4s 2024-08-25T19:47:50.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56100K .......... .......... .......... .......... .......... 33% 22.8M 4s 2024-08-25T19:47:50.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56150K .......... .......... .......... .......... .......... 33% 39.0M 4s 2024-08-25T19:47:50.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56200K .......... .......... .......... .......... .......... 33% 25.5M 4s 2024-08-25T19:47:50.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56250K .......... .......... .......... .......... .......... 33% 21.2M 4s 2024-08-25T19:47:50.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56300K .......... .......... .......... .......... .......... 33% 52.5M 4s 2024-08-25T19:47:50.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56350K .......... .......... .......... .......... .......... 33% 23.4M 4s 2024-08-25T19:47:50.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56400K .......... .......... .......... .......... .......... 33% 18.5M 4s 2024-08-25T19:47:50.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56450K .......... .......... .......... .......... .......... 33% 46.0M 4s 2024-08-25T19:47:50.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56500K .......... .......... .......... .......... .......... 33% 22.4M 4s 2024-08-25T19:47:50.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56550K .......... .......... .......... .......... .......... 33% 35.0M 4s 2024-08-25T19:47:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56600K .......... .......... .......... .......... .......... 33% 27.3M 4s 2024-08-25T19:47:50.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56650K .......... .......... .......... .......... .......... 33% 19.1M 4s 2024-08-25T19:47:50.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56700K .......... .......... .......... .......... .......... 33% 35.3M 4s 2024-08-25T19:47:50.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56750K .......... .......... .......... .......... .......... 33% 22.6M 4s 2024-08-25T19:47:50.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56800K .......... .......... .......... .......... .......... 33% 23.7M 4s 2024-08-25T19:47:50.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56850K .......... .......... .......... .......... .......... 33% 42.6M 4s 2024-08-25T19:47:50.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56900K .......... .......... .......... .......... .......... 33% 19.9M 4s 2024-08-25T19:47:50.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 56950K .......... .......... .......... .......... .......... 33% 41.6M 4s 2024-08-25T19:47:50.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57000K .......... .......... .......... .......... .......... 33% 22.4M 4s 2024-08-25T19:47:50.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57050K .......... .......... .......... .......... .......... 33% 48.8M 4s 2024-08-25T19:47:50.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57100K .......... .......... .......... .......... .......... 33% 22.5M 4s 2024-08-25T19:47:50.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57150K .......... .......... .......... .......... .......... 33% 16.8M 4s 2024-08-25T19:47:50.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57200K .......... .......... .......... .......... .......... 33% 22.5M 4s 2024-08-25T19:47:50.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57250K .......... .......... .......... .......... .......... 33% 44.0M 4s 2024-08-25T19:47:50.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57300K .......... .......... .......... .......... .......... 34% 24.4M 4s 2024-08-25T19:47:50.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57350K .......... .......... .......... .......... .......... 34% 39.1M 4s 2024-08-25T19:47:50.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57400K .......... .......... .......... .......... .......... 34% 15.9M 4s 2024-08-25T19:47:50.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57450K .......... .......... .......... .......... .......... 34% 42.0M 4s 2024-08-25T19:47:50.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57500K .......... .......... .......... .......... .......... 34% 22.8M 4s 2024-08-25T19:47:50.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57550K .......... .......... .......... .......... .......... 34% 24.1M 4s 2024-08-25T19:47:50.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57600K .......... .......... .......... .......... .......... 34% 34.7M 4s 2024-08-25T19:47:50.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57650K .......... .......... .......... .......... .......... 34% 28.1M 4s 2024-08-25T19:47:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57700K .......... .......... .......... .......... .......... 34% 32.8M 4s 2024-08-25T19:47:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57750K .......... .......... .......... .......... .......... 34% 28.5M 4s 2024-08-25T19:47:50.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57800K .......... .......... .......... .......... .......... 34% 17.1M 4s 2024-08-25T19:47:50.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57850K .......... .......... .......... .......... .......... 34% 30.4M 4s 2024-08-25T19:47:50.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57900K .......... .......... .......... .......... .......... 34% 28.2M 4s 2024-08-25T19:47:50.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 57950K .......... .......... .......... .......... .......... 34% 20.6M 4s 2024-08-25T19:47:50.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58000K .......... .......... .......... .......... .......... 34% 43.8M 4s 2024-08-25T19:47:50.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58050K .......... .......... .......... .......... .......... 34% 25.4M 4s 2024-08-25T19:47:50.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58100K .......... .......... .......... .......... .......... 34% 38.5M 4s 2024-08-25T19:47:50.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58150K .......... .......... .......... .......... .......... 34% 24.1M 4s 2024-08-25T19:47:50.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58200K .......... .......... .......... .......... .......... 34% 42.8M 4s 2024-08-25T19:47:50.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58250K .......... .......... .......... .......... .......... 34% 10.0M 4s 2024-08-25T19:47:50.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58300K .......... .......... .......... .......... .......... 34% 37.1M 4s 2024-08-25T19:47:50.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58350K .......... .......... .......... .......... .......... 34% 22.9M 4s 2024-08-25T19:47:50.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58400K .......... .......... .......... .......... .......... 34% 23.4M 4s 2024-08-25T19:47:50.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58450K .......... .......... .......... .......... .......... 34% 27.7M 4s 2024-08-25T19:47:50.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58500K .......... .......... .......... .......... .......... 34% 45.3M 4s 2024-08-25T19:47:50.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58550K .......... .......... .......... .......... .......... 34% 13.6M 4s 2024-08-25T19:47:50.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58600K .......... .......... .......... .......... .......... 34% 39.8M 4s 2024-08-25T19:47:50.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58650K .......... .......... .......... .......... .......... 34% 27.3M 4s 2024-08-25T19:47:50.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58700K .......... .......... .......... .......... .......... 34% 41.1M 4s 2024-08-25T19:47:50.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58750K .......... .......... .......... .......... .......... 34% 22.7M 4s 2024-08-25T19:47:50.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58800K .......... .......... .......... .......... .......... 34% 24.1M 4s 2024-08-25T19:47:50.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58850K .......... .......... .......... .......... .......... 34% 34.5M 4s 2024-08-25T19:47:50.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58900K .......... .......... .......... .......... .......... 34% 22.5M 4s 2024-08-25T19:47:50.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 58950K .......... .......... .......... .......... .......... 35% 21.1M 4s 2024-08-25T19:47:50.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59000K .......... .......... .......... .......... .......... 35% 40.2M 4s 2024-08-25T19:47:50.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59050K .......... .......... .......... .......... .......... 35% 13.8M 4s 2024-08-25T19:47:50.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59100K .......... .......... .......... .......... .......... 35% 33.0M 4s 2024-08-25T19:47:50.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59150K .......... .......... .......... .......... .......... 35% 20.1M 4s 2024-08-25T19:47:50.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59200K .......... .......... .......... .......... .......... 35% 37.8M 4s 2024-08-25T19:47:50.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59250K .......... .......... .......... .......... .......... 35% 39.6M 4s 2024-08-25T19:47:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59300K .......... .......... .......... .......... .......... 35% 24.9M 4s 2024-08-25T19:47:50.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59350K .......... .......... .......... .......... .......... 35% 30.3M 4s 2024-08-25T19:47:50.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59400K .......... .......... .......... .......... .......... 35% 27.3M 4s 2024-08-25T19:47:50.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59450K .......... .......... .......... .......... .......... 35% 62.7M 4s 2024-08-25T19:47:50.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59500K .......... .......... .......... .......... .......... 35% 22.5M 4s 2024-08-25T19:47:50.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59550K .......... .......... .......... .......... .......... 35% 21.8M 4s 2024-08-25T19:47:50.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59600K .......... .......... .......... .......... .......... 35% 21.7M 4s 2024-08-25T19:47:50.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59650K .......... .......... .......... .......... .......... 35% 37.9M 4s 2024-08-25T19:47:50.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59700K .......... .......... .......... .......... .......... 35% 23.6M 4s 2024-08-25T19:47:50.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59750K .......... .......... .......... .......... .......... 35% 34.4M 4s 2024-08-25T19:47:50.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59800K .......... .......... .......... .......... .......... 35% 25.7M 4s 2024-08-25T19:47:50.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59850K .......... .......... .......... .......... .......... 35% 42.3M 4s 2024-08-25T19:47:50.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59900K .......... .......... .......... .......... .......... 35% 22.1M 4s 2024-08-25T19:47:50.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 59950K .......... .......... .......... .......... .......... 35% 26.9M 4s 2024-08-25T19:47:50.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60000K .......... .......... .......... .......... .......... 35% 39.9M 4s 2024-08-25T19:47:50.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60050K .......... .......... .......... .......... .......... 35% 24.3M 4s 2024-08-25T19:47:50.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60100K .......... .......... .......... .......... .......... 35% 24.9M 4s 2024-08-25T19:47:50.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60150K .......... .......... .......... .......... .......... 35% 34.3M 4s 2024-08-25T19:47:50.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60200K .......... .......... .......... .......... .......... 35% 23.7M 4s 2024-08-25T19:47:50.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60250K .......... .......... .......... .......... .......... 35% 35.8M 4s 2024-08-25T19:47:50.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60300K .......... .......... .......... .......... .......... 35% 23.4M 4s 2024-08-25T19:47:50.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60350K .......... .......... .......... .......... .......... 35% 25.9M 4s 2024-08-25T19:47:50.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60400K .......... .......... .......... .......... .......... 35% 44.2M 4s 2024-08-25T19:47:50.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60450K .......... .......... .......... .......... .......... 35% 23.7M 4s 2024-08-25T19:47:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60500K .......... .......... .......... .......... .......... 35% 47.1M 4s 2024-08-25T19:47:50.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60550K .......... .......... .......... .......... .......... 35% 24.0M 4s 2024-08-25T19:47:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60600K .......... .......... .......... .......... .......... 35% 22.4M 4s 2024-08-25T19:47:50.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60650K .......... .......... .......... .......... .......... 36% 43.1M 4s 2024-08-25T19:47:50.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60700K .......... .......... .......... .......... .......... 36% 18.0M 4s 2024-08-25T19:47:50.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60750K .......... .......... .......... .......... .......... 36% 18.2M 4s 2024-08-25T19:47:50.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60800K .......... .......... .......... .......... .......... 36% 39.5M 4s 2024-08-25T19:47:50.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60850K .......... .......... .......... .......... .......... 36% 18.9M 4s 2024-08-25T19:47:50.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60900K .......... .......... .......... .......... .......... 36% 37.0M 4s 2024-08-25T19:47:50.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 60950K .......... .......... .......... .......... .......... 36% 28.2M 4s 2024-08-25T19:47:50.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61000K .......... .......... .......... .......... .......... 36% 35.2M 4s 2024-08-25T19:47:50.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61050K .......... .......... .......... .......... .......... 36% 24.9M 4s 2024-08-25T19:47:50.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61100K .......... .......... .......... .......... .......... 36% 51.1M 4s 2024-08-25T19:47:50.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61150K .......... .......... .......... .......... .......... 36% 12.0M 4s 2024-08-25T19:47:50.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61200K .......... .......... .......... .......... .......... 36% 56.7M 4s 2024-08-25T19:47:50.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61250K .......... .......... .......... .......... .......... 36% 21.5M 4s 2024-08-25T19:47:50.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61300K .......... .......... .......... .......... .......... 36% 39.7M 4s 2024-08-25T19:47:50.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61350K .......... .......... .......... .......... .......... 36% 22.9M 4s 2024-08-25T19:47:50.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61400K .......... .......... .......... .......... .......... 36% 48.0M 4s 2024-08-25T19:47:50.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61450K .......... .......... .......... .......... .......... 36% 24.1M 4s 2024-08-25T19:47:50.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61500K .......... .......... .......... .......... .......... 36% 42.4M 4s 2024-08-25T19:47:50.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61550K .......... .......... .......... .......... .......... 36% 25.7M 4s 2024-08-25T19:47:50.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61600K .......... .......... .......... .......... .......... 36% 20.1M 4s 2024-08-25T19:47:50.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61650K .......... .......... .......... .......... .......... 36% 34.0M 4s 2024-08-25T19:47:50.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61700K .......... .......... .......... .......... .......... 36% 15.2M 4s 2024-08-25T19:47:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61750K .......... .......... .......... .......... .......... 36% 22.1M 4s 2024-08-25T19:47:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61800K .......... .......... .......... .......... .......... 36% 51.4M 4s 2024-08-25T19:47:50.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61850K .......... .......... .......... .......... .......... 36% 13.3M 4s 2024-08-25T19:47:50.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61900K .......... .......... .......... .......... .......... 36% 34.8M 4s 2024-08-25T19:47:50.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 61950K .......... .......... .......... .......... .......... 36% 27.3M 4s 2024-08-25T19:47:50.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62000K .......... .......... .......... .......... .......... 36% 22.6M 4s 2024-08-25T19:47:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62050K .......... .......... .......... .......... .......... 36% 40.3M 4s 2024-08-25T19:47:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62100K .......... .......... .......... .......... .......... 36% 29.2M 4s 2024-08-25T19:47:50.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62150K .......... .......... .......... .......... .......... 36% 36.7M 4s 2024-08-25T19:47:50.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62200K .......... .......... .......... .......... .......... 36% 25.2M 4s 2024-08-25T19:47:50.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62250K .......... .......... .......... .......... .......... 36% 36.6M 4s 2024-08-25T19:47:50.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62300K .......... .......... .......... .......... .......... 36% 22.1M 4s 2024-08-25T19:47:50.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62350K .......... .......... .......... .......... .......... 37% 24.8M 4s 2024-08-25T19:47:50.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62400K .......... .......... .......... .......... .......... 37% 29.4M 4s 2024-08-25T19:47:50.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62450K .......... .......... .......... .......... .......... 37% 28.9M 4s 2024-08-25T19:47:50.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62500K .......... .......... .......... .......... .......... 37% 23.8M 4s 2024-08-25T19:47:50.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62550K .......... .......... .......... .......... .......... 37% 44.7M 4s 2024-08-25T19:47:50.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62600K .......... .......... .......... .......... .......... 37% 24.7M 4s 2024-08-25T19:47:50.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62650K .......... .......... .......... .......... .......... 37% 29.5M 4s 2024-08-25T19:47:50.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62700K .......... .......... .......... .......... .......... 37% 24.8M 4s 2024-08-25T19:47:50.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62750K .......... .......... .......... .......... .......... 37% 10.5M 4s 2024-08-25T19:47:50.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62800K .......... .......... .......... .......... .......... 37% 36.4M 4s 2024-08-25T19:47:50.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62850K .......... .......... .......... .......... .......... 37% 26.9M 4s 2024-08-25T19:47:50.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62900K .......... .......... .......... .......... .......... 37% 38.9M 4s 2024-08-25T19:47:50.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 62950K .......... .......... .......... .......... .......... 37% 23.1M 4s 2024-08-25T19:47:50.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63000K .......... .......... .......... .......... .......... 37% 17.6M 4s 2024-08-25T19:47:50.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63050K .......... .......... .......... .......... .......... 37% 44.0M 4s 2024-08-25T19:47:50.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63100K .......... .......... .......... .......... .......... 37% 26.2M 4s 2024-08-25T19:47:50.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63150K .......... .......... .......... .......... .......... 37% 18.2M 4s 2024-08-25T19:47:50.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63200K .......... .......... .......... .......... .......... 37% 40.2M 4s 2024-08-25T19:47:50.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63250K .......... .......... .......... .......... .......... 37% 21.7M 4s 2024-08-25T19:47:50.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63300K .......... .......... .......... .......... .......... 37% 38.6M 4s 2024-08-25T19:47:50.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63350K .......... .......... .......... .......... .......... 37% 24.1M 4s 2024-08-25T19:47:50.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63400K .......... .......... .......... .......... .......... 37% 48.8M 4s 2024-08-25T19:47:50.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63450K .......... .......... .......... .......... .......... 37% 16.6M 4s 2024-08-25T19:47:50.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63500K .......... .......... .......... .......... .......... 37% 30.1M 4s 2024-08-25T19:47:50.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63550K .......... .......... .......... .......... .......... 37% 23.9M 4s 2024-08-25T19:47:50.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63600K .......... .......... .......... .......... .......... 37% 38.2M 4s 2024-08-25T19:47:50.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63650K .......... .......... .......... .......... .......... 37% 20.2M 4s 2024-08-25T19:47:50.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63700K .......... .......... .......... .......... .......... 37% 43.1M 4s 2024-08-25T19:47:50.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63750K .......... .......... .......... .......... .......... 37% 23.7M 4s 2024-08-25T19:47:50.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63800K .......... .......... .......... .......... .......... 37% 45.1M 4s 2024-08-25T19:47:50.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63850K .......... .......... .......... .......... .......... 37% 23.5M 4s 2024-08-25T19:47:50.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63900K .......... .......... .......... .......... .......... 37% 43.6M 4s 2024-08-25T19:47:50.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 63950K .......... .......... .......... .......... .......... 37% 23.8M 4s 2024-08-25T19:47:50.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64000K .......... .......... .......... .......... .......... 37% 21.0M 4s 2024-08-25T19:47:50.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64050K .......... .......... .......... .......... .......... 38% 30.0M 4s 2024-08-25T19:47:50.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64100K .......... .......... .......... .......... .......... 38% 44.5M 4s 2024-08-25T19:47:50.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64150K .......... .......... .......... .......... .......... 38% 20.7M 4s 2024-08-25T19:47:50.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64200K .......... .......... .......... .......... .......... 38% 43.0M 4s 2024-08-25T19:47:50.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64250K .......... .......... .......... .......... .......... 38% 22.2M 4s 2024-08-25T19:47:50.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64300K .......... .......... .......... .......... .......... 38% 36.5M 4s 2024-08-25T19:47:50.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64350K .......... .......... .......... .......... .......... 38% 25.7M 4s 2024-08-25T19:47:50.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64400K .......... .......... .......... .......... .......... 38% 22.9M 4s 2024-08-25T19:47:50.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64450K .......... .......... .......... .......... .......... 38% 28.6M 4s 2024-08-25T19:47:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64500K .......... .......... .......... .......... .......... 38% 23.9M 4s 2024-08-25T19:47:50.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64550K .......... .......... .......... .......... .......... 38% 26.3M 4s 2024-08-25T19:47:50.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64600K .......... .......... .......... .......... .......... 38% 24.1M 4s 2024-08-25T19:47:50.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64650K .......... .......... .......... .......... .......... 38% 24.5M 4s 2024-08-25T19:47:50.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64700K .......... .......... .......... .......... .......... 38% 34.5M 4s 2024-08-25T19:47:50.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64750K .......... .......... .......... .......... .......... 38% 23.3M 4s 2024-08-25T19:47:50.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64800K .......... .......... .......... .......... .......... 38% 26.4M 4s 2024-08-25T19:47:50.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64850K .......... .......... .......... .......... .......... 38% 50.1M 4s 2024-08-25T19:47:50.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64900K .......... .......... .......... .......... .......... 38% 29.2M 4s 2024-08-25T19:47:50.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 64950K .......... .......... .......... .......... .......... 38% 22.3M 4s 2024-08-25T19:47:50.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65000K .......... .......... .......... .......... .......... 38% 23.5M 4s 2024-08-25T19:47:50.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65050K .......... .......... .......... .......... .......... 38% 52.0M 4s 2024-08-25T19:47:50.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65100K .......... .......... .......... .......... .......... 38% 23.8M 4s 2024-08-25T19:47:50.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65150K .......... .......... .......... .......... .......... 38% 20.6M 4s 2024-08-25T19:47:50.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65200K .......... .......... .......... .......... .......... 38% 25.7M 4s 2024-08-25T19:47:50.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65250K .......... .......... .......... .......... .......... 38% 41.1M 4s 2024-08-25T19:47:50.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65300K .......... .......... .......... .......... .......... 38% 20.6M 4s 2024-08-25T19:47:50.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65350K .......... .......... .......... .......... .......... 38% 43.9M 4s 2024-08-25T19:47:50.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65400K .......... .......... .......... .......... .......... 38% 16.3M 4s 2024-08-25T19:47:50.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65450K .......... .......... .......... .......... .......... 38% 47.8M 4s 2024-08-25T19:47:50.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65500K .......... .......... .......... .......... .......... 38% 24.0M 4s 2024-08-25T19:47:50.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65550K .......... .......... .......... .......... .......... 38% 24.5M 4s 2024-08-25T19:47:50.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65600K .......... .......... .......... .......... .......... 38% 38.8M 4s 2024-08-25T19:47:50.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65650K .......... .......... .......... .......... .......... 38% 26.1M 4s 2024-08-25T19:47:50.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65700K .......... .......... .......... .......... .......... 39% 22.8M 4s 2024-08-25T19:47:50.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65750K .......... .......... .......... .......... .......... 39% 43.9M 4s 2024-08-25T19:47:50.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65800K .......... .......... .......... .......... .......... 39% 17.8M 4s 2024-08-25T19:47:50.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65850K .......... .......... .......... .......... .......... 39% 11.4M 4s 2024-08-25T19:47:50.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65900K .......... .......... .......... .......... .......... 39% 192M 4s 2024-08-25T19:47:50.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 65950K .......... .......... .......... .......... .......... 39% 26.6M 4s 2024-08-25T19:47:50.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66000K .......... .......... .......... .......... .......... 39% 47.5M 4s 2024-08-25T19:47:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66050K .......... .......... .......... .......... .......... 39% 20.3M 4s 2024-08-25T19:47:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66100K .......... .......... .......... .......... .......... 39% 44.6M 4s 2024-08-25T19:47:50.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66150K .......... .......... .......... .......... .......... 39% 20.2M 4s 2024-08-25T19:47:50.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66200K .......... .......... .......... .......... .......... 39% 46.7M 4s 2024-08-25T19:47:50.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66250K .......... .......... .......... .......... .......... 39% 23.1M 4s 2024-08-25T19:47:50.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66300K .......... .......... .......... .......... .......... 39% 12.9M 4s 2024-08-25T19:47:50.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66350K .......... .......... .......... .......... .......... 39% 21.5M 4s 2024-08-25T19:47:50.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66400K .......... .......... .......... .......... .......... 39% 42.1M 4s 2024-08-25T19:47:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66450K .......... .......... .......... .......... .......... 39% 25.5M 4s 2024-08-25T19:47:50.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66500K .......... .......... .......... .......... .......... 39% 26.5M 4s 2024-08-25T19:47:50.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66550K .......... .......... .......... .......... .......... 39% 22.3M 4s 2024-08-25T19:47:50.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66600K .......... .......... .......... .......... .......... 39% 36.4M 4s 2024-08-25T19:47:50.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66650K .......... .......... .......... .......... .......... 39% 23.6M 4s 2024-08-25T19:47:50.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66700K .......... .......... .......... .......... .......... 39% 35.4M 4s 2024-08-25T19:47:50.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66750K .......... .......... .......... .......... .......... 39% 25.9M 4s 2024-08-25T19:47:50.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66800K .......... .......... .......... .......... .......... 39% 14.2M 4s 2024-08-25T19:47:50.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66850K .......... .......... .......... .......... .......... 39% 21.9M 4s 2024-08-25T19:47:50.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66900K .......... .......... .......... .......... .......... 39% 44.7M 4s 2024-08-25T19:47:50.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 66950K .......... .......... .......... .......... .......... 39% 17.2M 4s 2024-08-25T19:47:50.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67000K .......... .......... .......... .......... .......... 39% 50.0M 4s 2024-08-25T19:47:50.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67050K .......... .......... .......... .......... .......... 39% 22.3M 4s 2024-08-25T19:47:50.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67100K .......... .......... .......... .......... .......... 39% 48.6M 4s 2024-08-25T19:47:50.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67150K .......... .......... .......... .......... .......... 39% 19.7M 4s 2024-08-25T19:47:50.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67200K .......... .......... .......... .......... .......... 39% 23.6M 4s 2024-08-25T19:47:50.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67250K .......... .......... .......... .......... .......... 39% 19.7M 4s 2024-08-25T19:47:50.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67300K .......... .......... .......... .......... .......... 39% 23.5M 4s 2024-08-25T19:47:50.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67350K .......... .......... .......... .......... .......... 39% 48.7M 4s 2024-08-25T19:47:50.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67400K .......... .......... .......... .......... .......... 40% 24.4M 4s 2024-08-25T19:47:50.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67450K .......... .......... .......... .......... .......... 40% 23.0M 4s 2024-08-25T19:47:50.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67500K .......... .......... .......... .......... .......... 40% 42.5M 4s 2024-08-25T19:47:50.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67550K .......... .......... .......... .......... .......... 40% 22.6M 4s 2024-08-25T19:47:50.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67600K .......... .......... .......... .......... .......... 40% 25.2M 4s 2024-08-25T19:47:50.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67650K .......... .......... .......... .......... .......... 40% 40.4M 4s 2024-08-25T19:47:50.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67700K .......... .......... .......... .......... .......... 40% 23.0M 4s 2024-08-25T19:47:50.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67750K .......... .......... .......... .......... .......... 40% 40.8M 4s 2024-08-25T19:47:50.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67800K .......... .......... .......... .......... .......... 40% 21.4M 4s 2024-08-25T19:47:50.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67850K .......... .......... .......... .......... .......... 40% 60.6M 4s 2024-08-25T19:47:50.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67900K .......... .......... .......... .......... .......... 40% 22.7M 4s 2024-08-25T19:47:50.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 67950K .......... .......... .......... .......... .......... 40% 23.9M 4s 2024-08-25T19:47:50.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68000K .......... .......... .......... .......... .......... 40% 25.3M 4s 2024-08-25T19:47:51.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68050K .......... .......... .......... .......... .......... 40% 40.7M 4s 2024-08-25T19:47:51.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68100K .......... .......... .......... .......... .......... 40% 24.8M 4s 2024-08-25T19:47:51.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68150K .......... .......... .......... .......... .......... 40% 37.6M 4s 2024-08-25T19:47:51.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68200K .......... .......... .......... .......... .......... 40% 23.3M 4s 2024-08-25T19:47:51.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68250K .......... .......... .......... .......... .......... 40% 47.9M 4s 2024-08-25T19:47:51.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68300K .......... .......... .......... .......... .......... 40% 24.4M 4s 2024-08-25T19:47:51.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68350K .......... .......... .......... .......... .......... 40% 21.9M 4s 2024-08-25T19:47:51.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68400K .......... .......... .......... .......... .......... 40% 43.1M 4s 2024-08-25T19:47:51.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68450K .......... .......... .......... .......... .......... 40% 23.5M 4s 2024-08-25T19:47:51.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68500K .......... .......... .......... .......... .......... 40% 39.7M 4s 2024-08-25T19:47:51.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68550K .......... .......... .......... .......... .......... 40% 23.3M 4s 2024-08-25T19:47:51.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68600K .......... .......... .......... .......... .......... 40% 24.6M 4s 2024-08-25T19:47:51.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68650K .......... .......... .......... .......... .......... 40% 20.4M 4s 2024-08-25T19:47:51.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68700K .......... .......... .......... .......... .......... 40% 25.3M 4s 2024-08-25T19:47:51.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68750K .......... .......... .......... .......... .......... 40% 22.4M 4s 2024-08-25T19:47:51.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68800K .......... .......... .......... .......... .......... 40% 36.6M 4s 2024-08-25T19:47:51.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68850K .......... .......... .......... .......... .......... 40% 17.6M 4s 2024-08-25T19:47:51.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68900K .......... .......... .......... .......... .......... 40% 38.5M 4s 2024-08-25T19:47:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 68950K .......... .......... .......... .......... .......... 40% 26.2M 4s 2024-08-25T19:47:51.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69000K .......... .......... .......... .......... .......... 40% 37.2M 4s 2024-08-25T19:47:51.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69050K .......... .......... .......... .......... .......... 40% 27.3M 4s 2024-08-25T19:47:51.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69100K .......... .......... .......... .......... .......... 41% 21.3M 4s 2024-08-25T19:47:51.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69150K .......... .......... .......... .......... .......... 41% 18.9M 4s 2024-08-25T19:47:51.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69200K .......... .......... .......... .......... .......... 41% 46.7M 4s 2024-08-25T19:47:51.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69250K .......... .......... .......... .......... .......... 41% 31.6M 4s 2024-08-25T19:47:51.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69300K .......... .......... .......... .......... .......... 41% 27.1M 4s 2024-08-25T19:47:51.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69350K .......... .......... .......... .......... .......... 41% 23.1M 4s 2024-08-25T19:47:51.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69400K .......... .......... .......... .......... .......... 41% 42.0M 4s 2024-08-25T19:47:51.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69450K .......... .......... .......... .......... .......... 41% 18.7M 4s 2024-08-25T19:47:51.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69500K .......... .......... .......... .......... .......... 41% 38.8M 4s 2024-08-25T19:47:51.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69550K .......... .......... .......... .......... .......... 41% 22.1M 4s 2024-08-25T19:47:51.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69600K .......... .......... .......... .......... .......... 41% 26.1M 4s 2024-08-25T19:47:51.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69650K .......... .......... .......... .......... .......... 41% 18.6M 4s 2024-08-25T19:47:51.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69700K .......... .......... .......... .......... .......... 41% 38.6M 4s 2024-08-25T19:47:51.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69750K .......... .......... .......... .......... .......... 41% 27.1M 4s 2024-08-25T19:47:51.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69800K .......... .......... .......... .......... .......... 41% 14.7M 4s 2024-08-25T19:47:51.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69850K .......... .......... .......... .......... .......... 41% 24.5M 4s 2024-08-25T19:47:51.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69900K .......... .......... .......... .......... .......... 41% 53.7M 4s 2024-08-25T19:47:51.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 69950K .......... .......... .......... .......... .......... 41% 20.5M 4s 2024-08-25T19:47:51.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70000K .......... .......... .......... .......... .......... 41% 24.9M 4s 2024-08-25T19:47:51.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70050K .......... .......... .......... .......... .......... 41% 35.3M 4s 2024-08-25T19:47:51.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70100K .......... .......... .......... .......... .......... 41% 26.5M 4s 2024-08-25T19:47:51.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70150K .......... .......... .......... .......... .......... 41% 33.3M 4s 2024-08-25T19:47:51.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70200K .......... .......... .......... .......... .......... 41% 28.3M 4s 2024-08-25T19:47:51.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70250K .......... .......... .......... .......... .......... 41% 22.9M 4s 2024-08-25T19:47:51.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70300K .......... .......... .......... .......... .......... 41% 41.2M 4s 2024-08-25T19:47:51.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70350K .......... .......... .......... .......... .......... 41% 18.1M 4s 2024-08-25T19:47:51.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70400K .......... .......... .......... .......... .......... 41% 21.8M 4s 2024-08-25T19:47:51.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70450K .......... .......... .......... .......... .......... 41% 39.7M 4s 2024-08-25T19:47:51.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70500K .......... .......... .......... .......... .......... 41% 24.6M 4s 2024-08-25T19:47:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70550K .......... .......... .......... .......... .......... 41% 38.9M 4s 2024-08-25T19:47:51.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70600K .......... .......... .......... .......... .......... 41% 11.3M 4s 2024-08-25T19:47:51.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70650K .......... .......... .......... .......... .......... 41% 46.2M 4s 2024-08-25T19:47:51.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70700K .......... .......... .......... .......... .......... 41% 33.2M 4s 2024-08-25T19:47:51.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70750K .......... .......... .......... .......... .......... 42% 18.5M 4s 2024-08-25T19:47:51.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70800K .......... .......... .......... .......... .......... 42% 44.1M 4s 2024-08-25T19:47:51.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70850K .......... .......... .......... .......... .......... 42% 19.3M 4s 2024-08-25T19:47:51.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70900K .......... .......... .......... .......... .......... 42% 27.4M 4s 2024-08-25T19:47:51.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 70950K .......... .......... .......... .......... .......... 42% 44.9M 4s 2024-08-25T19:47:51.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71000K .......... .......... .......... .......... .......... 42% 23.8M 4s 2024-08-25T19:47:51.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71050K .......... .......... .......... .......... .......... 42% 37.6M 4s 2024-08-25T19:47:51.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71100K .......... .......... .......... .......... .......... 42% 26.4M 4s 2024-08-25T19:47:51.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71150K .......... .......... .......... .......... .......... 42% 14.7M 4s 2024-08-25T19:47:51.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71200K .......... .......... .......... .......... .......... 42% 51.5M 4s 2024-08-25T19:47:51.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71250K .......... .......... .......... .......... .......... 42% 22.4M 4s 2024-08-25T19:47:51.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71300K .......... .......... .......... .......... .......... 42% 35.5M 4s 2024-08-25T19:47:51.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71350K .......... .......... .......... .......... .......... 42% 25.5M 4s 2024-08-25T19:47:51.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71400K .......... .......... .......... .......... .......... 42% 21.8M 4s 2024-08-25T19:47:51.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71450K .......... .......... .......... .......... .......... 42% 39.5M 4s 2024-08-25T19:47:51.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71500K .......... .......... .......... .......... .......... 42% 23.0M 4s 2024-08-25T19:47:51.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71550K .......... .......... .......... .......... .......... 42% 27.5M 4s 2024-08-25T19:47:51.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71600K .......... .......... .......... .......... .......... 42% 39.9M 4s 2024-08-25T19:47:51.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71650K .......... .......... .......... .......... .......... 42% 15.4M 4s 2024-08-25T19:47:51.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71700K .......... .......... .......... .......... .......... 42% 44.2M 4s 2024-08-25T19:47:51.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71750K .......... .......... .......... .......... .......... 42% 23.5M 4s 2024-08-25T19:47:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71800K .......... .......... .......... .......... .......... 42% 34.6M 4s 2024-08-25T19:47:51.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71850K .......... .......... .......... .......... .......... 42% 27.7M 4s 2024-08-25T19:47:51.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71900K .......... .......... .......... .......... .......... 42% 46.8M 4s 2024-08-25T19:47:51.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 71950K .......... .......... .......... .......... .......... 42% 15.9M 4s 2024-08-25T19:47:51.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72000K .......... .......... .......... .......... .......... 42% 44.2M 4s 2024-08-25T19:47:51.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72050K .......... .......... .......... .......... .......... 42% 26.5M 4s 2024-08-25T19:47:51.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72100K .......... .......... .......... .......... .......... 42% 37.4M 4s 2024-08-25T19:47:51.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72150K .......... .......... .......... .......... .......... 42% 23.8M 4s 2024-08-25T19:47:51.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72200K .......... .......... .......... .......... .......... 42% 33.8M 4s 2024-08-25T19:47:51.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72250K .......... .......... .......... .......... .......... 42% 28.9M 4s 2024-08-25T19:47:51.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72300K .......... .......... .......... .......... .......... 42% 31.9M 4s 2024-08-25T19:47:51.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72350K .......... .......... .......... .......... .......... 42% 27.3M 4s 2024-08-25T19:47:51.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72400K .......... .......... .......... .......... .......... 42% 21.8M 4s 2024-08-25T19:47:51.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72450K .......... .......... .......... .......... .......... 43% 35.0M 4s 2024-08-25T19:47:51.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72500K .......... .......... .......... .......... .......... 43% 18.4M 4s 2024-08-25T19:47:51.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72550K .......... .......... .......... .......... .......... 43% 21.2M 4s 2024-08-25T19:47:51.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72600K .......... .......... .......... .......... .......... 43% 48.4M 4s 2024-08-25T19:47:51.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72650K .......... .......... .......... .......... .......... 43% 23.8M 4s 2024-08-25T19:47:51.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72700K .......... .......... .......... .......... .......... 43% 44.7M 4s 2024-08-25T19:47:51.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72750K .......... .......... .......... .......... .......... 43% 24.6M 4s 2024-08-25T19:47:51.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72800K .......... .......... .......... .......... .......... 43% 24.3M 4s 2024-08-25T19:47:51.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72850K .......... .......... .......... .......... .......... 43% 42.2M 4s 2024-08-25T19:47:51.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72900K .......... .......... .......... .......... .......... 43% 24.2M 4s 2024-08-25T19:47:51.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 72950K .......... .......... .......... .......... .......... 43% 40.4M 4s 2024-08-25T19:47:51.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73000K .......... .......... .......... .......... .......... 43% 24.1M 4s 2024-08-25T19:47:51.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73050K .......... .......... .......... .......... .......... 43% 42.9M 4s 2024-08-25T19:47:51.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73100K .......... .......... .......... .......... .......... 43% 23.8M 4s 2024-08-25T19:47:51.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73150K .......... .......... .......... .......... .......... 43% 16.2M 4s 2024-08-25T19:47:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73200K .......... .......... .......... .......... .......... 43% 26.6M 4s 2024-08-25T19:47:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73250K .......... .......... .......... .......... .......... 43% 44.4M 4s 2024-08-25T19:47:51.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73300K .......... .......... .......... .......... .......... 43% 24.5M 4s 2024-08-25T19:47:51.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73350K .......... .......... .......... .......... .......... 43% 43.2M 4s 2024-08-25T19:47:51.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73400K .......... .......... .......... .......... .......... 43% 23.1M 4s 2024-08-25T19:47:51.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73450K .......... .......... .......... .......... .......... 43% 39.8M 4s 2024-08-25T19:47:51.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73500K .......... .......... .......... .......... .......... 43% 26.5M 4s 2024-08-25T19:47:51.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73550K .......... .......... .......... .......... .......... 43% 23.8M 4s 2024-08-25T19:47:51.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73600K .......... .......... .......... .......... .......... 43% 23.4M 4s 2024-08-25T19:47:51.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73650K .......... .......... .......... .......... .......... 43% 45.1M 4s 2024-08-25T19:47:51.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73700K .......... .......... .......... .......... .......... 43% 22.9M 4s 2024-08-25T19:47:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73750K .......... .......... .......... .......... .......... 43% 44.4M 4s 2024-08-25T19:47:51.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73800K .......... .......... .......... .......... .......... 43% 23.0M 4s 2024-08-25T19:47:51.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73850K .......... .......... .......... .......... .......... 43% 49.9M 4s 2024-08-25T19:47:51.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73900K .......... .......... .......... .......... .......... 43% 30.0M 4s 2024-08-25T19:47:51.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 73950K .......... .......... .......... .......... .......... 43% 19.6M 4s 2024-08-25T19:47:51.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74000K .......... .......... .......... .......... .......... 43% 43.9M 4s 2024-08-25T19:47:51.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74050K .......... .......... .......... .......... .......... 43% 24.5M 4s 2024-08-25T19:47:51.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74100K .......... .......... .......... .......... .......... 43% 37.2M 4s 2024-08-25T19:47:51.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74150K .......... .......... .......... .......... .......... 44% 16.6M 4s 2024-08-25T19:47:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74200K .......... .......... .......... .......... .......... 44% 22.2M 4s 2024-08-25T19:47:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74250K .......... .......... .......... .......... .......... 44% 43.7M 4s 2024-08-25T19:47:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74300K .......... .......... .......... .......... .......... 44% 22.5M 4s 2024-08-25T19:47:51.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74350K .......... .......... .......... .......... .......... 44% 20.2M 4s 2024-08-25T19:47:51.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74400K .......... .......... .......... .......... .......... 44% 43.8M 4s 2024-08-25T19:47:51.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74450K .......... .......... .......... .......... .......... 44% 21.4M 4s 2024-08-25T19:47:51.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74500K .......... .......... .......... .......... .......... 44% 17.6M 4s 2024-08-25T19:47:51.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74550K .......... .......... .......... .......... .......... 44% 32.1M 4s 2024-08-25T19:47:51.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74600K .......... .......... .......... .......... .......... 44% 29.3M 4s 2024-08-25T19:47:51.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74650K .......... .......... .......... .......... .......... 44% 26.0M 4s 2024-08-25T19:47:51.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74700K .......... .......... .......... .......... .......... 44% 42.3M 4s 2024-08-25T19:47:51.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74750K .......... .......... .......... .......... .......... 44% 15.8M 4s 2024-08-25T19:47:51.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74800K .......... .......... .......... .......... .......... 44% 45.8M 4s 2024-08-25T19:47:51.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74850K .......... .......... .......... .......... .......... 44% 20.7M 4s 2024-08-25T19:47:51.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74900K .......... .......... .......... .......... .......... 44% 28.1M 4s 2024-08-25T19:47:51.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 74950K .......... .......... .......... .......... .......... 44% 23.7M 4s 2024-08-25T19:47:51.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75000K .......... .......... .......... .......... .......... 44% 27.0M 4s 2024-08-25T19:47:51.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75050K .......... .......... .......... .......... .......... 44% 15.6M 4s 2024-08-25T19:47:51.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75100K .......... .......... .......... .......... .......... 44% 36.1M 4s 2024-08-25T19:47:51.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75150K .......... .......... .......... .......... .......... 44% 19.0M 4s 2024-08-25T19:47:51.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75200K .......... .......... .......... .......... .......... 44% 22.5M 4s 2024-08-25T19:47:51.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75250K .......... .......... .......... .......... .......... 44% 38.4M 3s 2024-08-25T19:47:51.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75300K .......... .......... .......... .......... .......... 44% 28.2M 3s 2024-08-25T19:47:51.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75350K .......... .......... .......... .......... .......... 44% 36.8M 3s 2024-08-25T19:47:51.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75400K .......... .......... .......... .......... .......... 44% 23.8M 3s 2024-08-25T19:47:51.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75450K .......... .......... .......... .......... .......... 44% 23.8M 3s 2024-08-25T19:47:51.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75500K .......... .......... .......... .......... .......... 44% 58.4M 3s 2024-08-25T19:47:51.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75550K .......... .......... .......... .......... .......... 44% 21.5M 3s 2024-08-25T19:47:51.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75600K .......... .......... .......... .......... .......... 44% 27.6M 3s 2024-08-25T19:47:51.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75650K .......... .......... .......... .......... .......... 44% 34.1M 3s 2024-08-25T19:47:51.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75700K .......... .......... .......... .......... .......... 44% 27.5M 3s 2024-08-25T19:47:51.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75750K .......... .......... .......... .......... .......... 44% 41.3M 3s 2024-08-25T19:47:51.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75800K .......... .......... .......... .......... .......... 44% 24.2M 3s 2024-08-25T19:47:51.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75850K .......... .......... .......... .......... .......... 45% 39.9M 3s 2024-08-25T19:47:51.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75900K .......... .......... .......... .......... .......... 45% 24.4M 3s 2024-08-25T19:47:51.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 75950K .......... .......... .......... .......... .......... 45% 16.9M 3s 2024-08-25T19:47:51.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76000K .......... .......... .......... .......... .......... 45% 13.7M 3s 2024-08-25T19:47:51.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76050K .......... .......... .......... .......... .......... 45% 50.6M 3s 2024-08-25T19:47:51.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76100K .......... .......... .......... .......... .......... 45% 31.3M 3s 2024-08-25T19:47:51.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76150K .......... .......... .......... .......... .......... 45% 29.3M 3s 2024-08-25T19:47:51.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76200K .......... .......... .......... .......... .......... 45% 24.1M 3s 2024-08-25T19:47:51.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76250K .......... .......... .......... .......... .......... 45% 45.9M 3s 2024-08-25T19:47:51.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76300K .......... .......... .......... .......... .......... 45% 23.1M 3s 2024-08-25T19:47:51.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76350K .......... .......... .......... .......... .......... 45% 23.4M 3s 2024-08-25T19:47:51.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76400K .......... .......... .......... .......... .......... 45% 31.8M 3s 2024-08-25T19:47:51.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76450K .......... .......... .......... .......... .......... 45% 21.5M 3s 2024-08-25T19:47:51.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76500K .......... .......... .......... .......... .......... 45% 23.1M 3s 2024-08-25T19:47:51.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76550K .......... .......... .......... .......... .......... 45% 31.9M 3s 2024-08-25T19:47:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76600K .......... .......... .......... .......... .......... 45% 23.3M 3s 2024-08-25T19:47:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76650K .......... .......... .......... .......... .......... 45% 42.5M 3s 2024-08-25T19:47:51.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76700K .......... .......... .......... .......... .......... 45% 24.0M 3s 2024-08-25T19:47:51.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76750K .......... .......... .......... .......... .......... 45% 23.3M 3s 2024-08-25T19:47:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76800K .......... .......... .......... .......... .......... 45% 33.1M 3s 2024-08-25T19:47:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76850K .......... .......... .......... .......... .......... 45% 22.8M 3s 2024-08-25T19:47:51.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76900K .......... .......... .......... .......... .......... 45% 29.7M 3s 2024-08-25T19:47:51.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 76950K .......... .......... .......... .......... .......... 45% 25.0M 3s 2024-08-25T19:47:51.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77000K .......... .......... .......... .......... .......... 45% 45.3M 3s 2024-08-25T19:47:51.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77050K .......... .......... .......... .......... .......... 45% 21.8M 3s 2024-08-25T19:47:51.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77100K .......... .......... .......... .......... .......... 45% 23.4M 3s 2024-08-25T19:47:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77150K .......... .......... .......... .......... .......... 45% 27.3M 3s 2024-08-25T19:47:51.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77200K .......... .......... .......... .......... .......... 45% 35.0M 3s 2024-08-25T19:47:51.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77250K .......... .......... .......... .......... .......... 45% 26.3M 3s 2024-08-25T19:47:51.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77300K .......... .......... .......... .......... .......... 45% 46.7M 3s 2024-08-25T19:47:51.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77350K .......... .......... .......... .......... .......... 45% 23.7M 3s 2024-08-25T19:47:51.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77400K .......... .......... .......... .......... .......... 45% 41.0M 3s 2024-08-25T19:47:51.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77450K .......... .......... .......... .......... .......... 45% 28.1M 3s 2024-08-25T19:47:51.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77500K .......... .......... .......... .......... .......... 46% 29.3M 3s 2024-08-25T19:47:51.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77550K .......... .......... .......... .......... .......... 46% 25.1M 3s 2024-08-25T19:47:51.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77600K .......... .......... .......... .......... .......... 46% 23.6M 3s 2024-08-25T19:47:51.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77650K .......... .......... .......... .......... .......... 46% 19.7M 3s 2024-08-25T19:47:51.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77700K .......... .......... .......... .......... .......... 46% 45.8M 3s 2024-08-25T19:47:51.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77750K .......... .......... .......... .......... .......... 46% 22.7M 3s 2024-08-25T19:47:51.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77800K .......... .......... .......... .......... .......... 46% 44.0M 3s 2024-08-25T19:47:51.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77850K .......... .......... .......... .......... .......... 46% 23.1M 3s 2024-08-25T19:47:51.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77900K .......... .......... .......... .......... .......... 46% 42.5M 3s 2024-08-25T19:47:51.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 77950K .......... .......... .......... .......... .......... 46% 23.4M 3s 2024-08-25T19:47:51.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78000K .......... .......... .......... .......... .......... 46% 25.2M 3s 2024-08-25T19:47:51.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78050K .......... .......... .......... .......... .......... 46% 37.2M 3s 2024-08-25T19:47:51.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78100K .......... .......... .......... .......... .......... 46% 14.0M 3s 2024-08-25T19:47:51.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78150K .......... .......... .......... .......... .......... 46% 24.3M 3s 2024-08-25T19:47:51.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78200K .......... .......... .......... .......... .......... 46% 39.4M 3s 2024-08-25T19:47:51.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78250K .......... .......... .......... .......... .......... 46% 24.8M 3s 2024-08-25T19:47:51.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78300K .......... .......... .......... .......... .......... 46% 37.9M 3s 2024-08-25T19:47:51.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78350K .......... .......... .......... .......... .......... 46% 22.9M 3s 2024-08-25T19:47:51.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78400K .......... .......... .......... .......... .......... 46% 23.4M 3s 2024-08-25T19:47:51.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78450K .......... .......... .......... .......... .......... 46% 47.8M 3s 2024-08-25T19:47:51.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78500K .......... .......... .......... .......... .......... 46% 20.2M 3s 2024-08-25T19:47:51.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78550K .......... .......... .......... .......... .......... 46% 34.2M 3s 2024-08-25T19:47:51.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78600K .......... .......... .......... .......... .......... 46% 24.0M 3s 2024-08-25T19:47:51.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78650K .......... .......... .......... .......... .......... 46% 45.2M 3s 2024-08-25T19:47:51.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78700K .......... .......... .......... .......... .......... 46% 23.1M 3s 2024-08-25T19:47:51.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78750K .......... .......... .......... .......... .......... 46% 22.2M 3s 2024-08-25T19:47:51.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78800K .......... .......... .......... .......... .......... 46% 20.3M 3s 2024-08-25T19:47:51.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78850K .......... .......... .......... .......... .......... 46% 38.3M 3s 2024-08-25T19:47:51.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78900K .......... .......... .......... .......... .......... 46% 21.0M 3s 2024-08-25T19:47:51.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 78950K .......... .......... .......... .......... .......... 46% 41.3M 3s 2024-08-25T19:47:51.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79000K .......... .......... .......... .......... .......... 46% 24.8M 3s 2024-08-25T19:47:51.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79050K .......... .......... .......... .......... .......... 46% 46.7M 3s 2024-08-25T19:47:51.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79100K .......... .......... .......... .......... .......... 46% 23.9M 3s 2024-08-25T19:47:51.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79150K .......... .......... .......... .......... .......... 46% 25.8M 3s 2024-08-25T19:47:51.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79200K .......... .......... .......... .......... .......... 47% 30.0M 3s 2024-08-25T19:47:51.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79250K .......... .......... .......... .......... .......... 47% 33.4M 3s 2024-08-25T19:47:51.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79300K .......... .......... .......... .......... .......... 47% 19.8M 3s 2024-08-25T19:47:51.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79350K .......... .......... .......... .......... .......... 47% 26.5M 3s 2024-08-25T19:47:51.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79400K .......... .......... .......... .......... .......... 47% 24.2M 3s 2024-08-25T19:47:51.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79450K .......... .......... .......... .......... .......... 47% 21.5M 3s 2024-08-25T19:47:51.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79500K .......... .......... .......... .......... .......... 47% 14.6M 3s 2024-08-25T19:47:51.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79550K .......... .......... .......... .......... .......... 47% 23.5M 3s 2024-08-25T19:47:51.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79600K .......... .......... .......... .......... .......... 47% 50.4M 3s 2024-08-25T19:47:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79650K .......... .......... .......... .......... .......... 47% 25.0M 3s 2024-08-25T19:47:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79700K .......... .......... .......... .......... .......... 47% 38.3M 3s 2024-08-25T19:47:51.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79750K .......... .......... .......... .......... .......... 47% 22.7M 3s 2024-08-25T19:47:51.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79800K .......... .......... .......... .......... .......... 47% 51.4M 3s 2024-08-25T19:47:51.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79850K .......... .......... .......... .......... .......... 47% 16.2M 3s 2024-08-25T19:47:51.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79900K .......... .......... .......... .......... .......... 47% 23.2M 3s 2024-08-25T19:47:51.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 79950K .......... .......... .......... .......... .......... 47% 23.8M 3s 2024-08-25T19:47:51.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80000K .......... .......... .......... .......... .......... 47% 42.6M 3s 2024-08-25T19:47:51.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80050K .......... .......... .......... .......... .......... 47% 24.4M 3s 2024-08-25T19:47:51.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80100K .......... .......... .......... .......... .......... 47% 42.2M 3s 2024-08-25T19:47:51.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80150K .......... .......... .......... .......... .......... 47% 22.8M 3s 2024-08-25T19:47:51.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80200K .......... .......... .......... .......... .......... 47% 49.6M 3s 2024-08-25T19:47:51.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80250K .......... .......... .......... .......... .......... 47% 21.2M 3s 2024-08-25T19:47:51.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80300K .......... .......... .......... .......... .......... 47% 59.3M 3s 2024-08-25T19:47:51.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80350K .......... .......... .......... .......... .......... 47% 22.6M 3s 2024-08-25T19:47:51.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80400K .......... .......... .......... .......... .......... 47% 24.0M 3s 2024-08-25T19:47:51.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80450K .......... .......... .......... .......... .......... 47% 40.3M 3s 2024-08-25T19:47:51.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80500K .......... .......... .......... .......... .......... 47% 24.4M 3s 2024-08-25T19:47:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80550K .......... .......... .......... .......... .......... 47% 21.9M 3s 2024-08-25T19:47:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80600K .......... .......... .......... .......... .......... 47% 52.4M 3s 2024-08-25T19:47:51.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80650K .......... .......... .......... .......... .......... 47% 28.8M 3s 2024-08-25T19:47:51.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80700K .......... .......... .......... .......... .......... 47% 30.2M 3s 2024-08-25T19:47:51.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80750K .......... .......... .......... .......... .......... 47% 23.8M 3s 2024-08-25T19:47:51.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80800K .......... .......... .......... .......... .......... 47% 41.6M 3s 2024-08-25T19:47:51.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80850K .......... .......... .......... .......... .......... 47% 26.0M 3s 2024-08-25T19:47:51.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80900K .......... .......... .......... .......... .......... 48% 20.9M 3s 2024-08-25T19:47:51.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 80950K .......... .......... .......... .......... .......... 48% 44.1M 3s 2024-08-25T19:47:51.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81000K .......... .......... .......... .......... .......... 48% 12.8M 3s 2024-08-25T19:47:51.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81050K .......... .......... .......... .......... .......... 48% 28.3M 3s 2024-08-25T19:47:51.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81100K .......... .......... .......... .......... .......... 48% 53.1M 3s 2024-08-25T19:47:51.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81150K .......... .......... .......... .......... .......... 48% 23.4M 3s 2024-08-25T19:47:51.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81200K .......... .......... .......... .......... .......... 48% 24.9M 3s 2024-08-25T19:47:51.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81250K .......... .......... .......... .......... .......... 48% 49.1M 3s 2024-08-25T19:47:51.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81300K .......... .......... .......... .......... .......... 48% 30.6M 3s 2024-08-25T19:47:51.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81350K .......... .......... .......... .......... .......... 48% 24.2M 3s 2024-08-25T19:47:51.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81400K .......... .......... .......... .......... .......... 48% 31.0M 3s 2024-08-25T19:47:51.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81450K .......... .......... .......... .......... .......... 48% 29.9M 3s 2024-08-25T19:47:51.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81500K .......... .......... .......... .......... .......... 48% 13.1M 3s 2024-08-25T19:47:51.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81550K .......... .......... .......... .......... .......... 48% 20.2M 3s 2024-08-25T19:47:51.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81600K .......... .......... .......... .......... .......... 48% 24.2M 3s 2024-08-25T19:47:51.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81650K .......... .......... .......... .......... .......... 48% 43.7M 3s 2024-08-25T19:47:51.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81700K .......... .......... .......... .......... .......... 48% 24.9M 3s 2024-08-25T19:47:51.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81750K .......... .......... .......... .......... .......... 48% 61.7M 3s 2024-08-25T19:47:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81800K .......... .......... .......... .......... .......... 48% 19.1M 3s 2024-08-25T19:47:51.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81850K .......... .......... .......... .......... .......... 48% 27.8M 3s 2024-08-25T19:47:51.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81900K .......... .......... .......... .......... .......... 48% 28.6M 3s 2024-08-25T19:47:51.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 81950K .......... .......... .......... .......... .......... 48% 22.4M 3s 2024-08-25T19:47:51.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82000K .......... .......... .......... .......... .......... 48% 39.0M 3s 2024-08-25T19:47:51.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82050K .......... .......... .......... .......... .......... 48% 23.5M 3s 2024-08-25T19:47:51.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82100K .......... .......... .......... .......... .......... 48% 36.6M 3s 2024-08-25T19:47:51.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82150K .......... .......... .......... .......... .......... 48% 23.1M 3s 2024-08-25T19:47:51.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82200K .......... .......... .......... .......... .......... 48% 19.0M 3s 2024-08-25T19:47:51.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82250K .......... .......... .......... .......... .......... 48% 43.2M 3s 2024-08-25T19:47:51.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82300K .......... .......... .......... .......... .......... 48% 22.8M 3s 2024-08-25T19:47:51.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82350K .......... .......... .......... .......... .......... 48% 24.5M 3s 2024-08-25T19:47:51.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82400K .......... .......... .......... .......... .......... 48% 41.4M 3s 2024-08-25T19:47:51.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82450K .......... .......... .......... .......... .......... 48% 29.1M 3s 2024-08-25T19:47:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82500K .......... .......... .......... .......... .......... 48% 32.0M 3s 2024-08-25T19:47:51.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82550K .......... .......... .......... .......... .......... 49% 25.2M 3s 2024-08-25T19:47:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82600K .......... .......... .......... .......... .......... 49% 26.3M 3s 2024-08-25T19:47:51.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82650K .......... .......... .......... .......... .......... 49% 23.8M 3s 2024-08-25T19:47:51.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82700K .......... .......... .......... .......... .......... 49% 40.5M 3s 2024-08-25T19:47:51.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82750K .......... .......... .......... .......... .......... 49% 21.3M 3s 2024-08-25T19:47:51.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82800K .......... .......... .......... .......... .......... 49% 26.4M 3s 2024-08-25T19:47:51.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82850K .......... .......... .......... .......... .......... 49% 18.7M 3s 2024-08-25T19:47:51.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82900K .......... .......... .......... .......... .......... 49% 29.3M 3s 2024-08-25T19:47:51.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 82950K .......... .......... .......... .......... .......... 49% 24.3M 3s 2024-08-25T19:47:51.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83000K .......... .......... .......... .......... .......... 49% 38.3M 3s 2024-08-25T19:47:51.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83050K .......... .......... .......... .......... .......... 49% 23.6M 3s 2024-08-25T19:47:51.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83100K .......... .......... .......... .......... .......... 49% 40.4M 3s 2024-08-25T19:47:51.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83150K .......... .......... .......... .......... .......... 49% 25.9M 3s 2024-08-25T19:47:51.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83200K .......... .......... .......... .......... .......... 49% 24.3M 3s 2024-08-25T19:47:51.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83250K .......... .......... .......... .......... .......... 49% 44.6M 3s 2024-08-25T19:47:51.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83300K .......... .......... .......... .......... .......... 49% 23.5M 3s 2024-08-25T19:47:51.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83350K .......... .......... .......... .......... .......... 49% 24.1M 3s 2024-08-25T19:47:51.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83400K .......... .......... .......... .......... .......... 49% 44.8M 3s 2024-08-25T19:47:51.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83450K .......... .......... .......... .......... .......... 49% 23.0M 3s 2024-08-25T19:47:51.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83500K .......... .......... .......... .......... .......... 49% 39.2M 3s 2024-08-25T19:47:51.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83550K .......... .......... .......... .......... .......... 49% 26.2M 3s 2024-08-25T19:47:51.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83600K .......... .......... .......... .......... .......... 49% 25.0M 3s 2024-08-25T19:47:51.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83650K .......... .......... .......... .......... .......... 49% 43.9M 3s 2024-08-25T19:47:51.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83700K .......... .......... .......... .......... .......... 49% 25.4M 3s 2024-08-25T19:47:51.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83750K .......... .......... .......... .......... .......... 49% 31.7M 3s 2024-08-25T19:47:51.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83800K .......... .......... .......... .......... .......... 49% 21.5M 3s 2024-08-25T19:47:51.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83850K .......... .......... .......... .......... .......... 49% 20.6M 3s 2024-08-25T19:47:51.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83900K .......... .......... .......... .......... .......... 49% 59.7M 3s 2024-08-25T19:47:51.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 83950K .......... .......... .......... .......... .......... 49% 20.2M 3s 2024-08-25T19:47:51.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84000K .......... .......... .......... .......... .......... 49% 20.6M 3s 2024-08-25T19:47:51.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84050K .......... .......... .......... .......... .......... 49% 41.7M 3s 2024-08-25T19:47:51.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84100K .......... .......... .......... .......... .......... 49% 25.2M 3s 2024-08-25T19:47:51.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84150K .......... .......... .......... .......... .......... 49% 38.0M 3s 2024-08-25T19:47:51.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84200K .......... .......... .......... .......... .......... 49% 23.6M 3s 2024-08-25T19:47:51.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84250K .......... .......... .......... .......... .......... 50% 37.2M 3s 2024-08-25T19:47:51.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84300K .......... .......... .......... .......... .......... 50% 24.7M 3s 2024-08-25T19:47:51.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84350K .......... .......... .......... .......... .......... 50% 29.8M 3s 2024-08-25T19:47:51.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84400K .......... .......... .......... .......... .......... 50% 19.3M 3s 2024-08-25T19:47:51.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84450K .......... .......... .......... .......... .......... 50% 28.5M 3s 2024-08-25T19:47:51.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84500K .......... .......... .......... .......... .......... 50% 16.8M 3s 2024-08-25T19:47:51.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84550K .......... .......... .......... .......... .......... 50% 31.2M 3s 2024-08-25T19:47:51.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84600K .......... .......... .......... .......... .......... 50% 25.5M 3s 2024-08-25T19:47:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84650K .......... .......... .......... .......... .......... 50% 40.8M 3s 2024-08-25T19:47:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84700K .......... .......... .......... .......... .......... 50% 22.1M 3s 2024-08-25T19:47:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84750K .......... .......... .......... .......... .......... 50% 19.8M 3s 2024-08-25T19:47:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84800K .......... .......... .......... .......... .......... 50% 44.1M 3s 2024-08-25T19:47:51.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84850K .......... .......... .......... .......... .......... 50% 25.6M 3s 2024-08-25T19:47:51.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84900K .......... .......... .......... .......... .......... 50% 42.5M 3s 2024-08-25T19:47:51.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 84950K .......... .......... .......... .......... .......... 50% 21.0M 3s 2024-08-25T19:47:51.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85000K .......... .......... .......... .......... .......... 50% 23.4M 3s 2024-08-25T19:47:51.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85050K .......... .......... .......... .......... .......... 50% 53.8M 3s 2024-08-25T19:47:51.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85100K .......... .......... .......... .......... .......... 50% 19.7M 3s 2024-08-25T19:47:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85150K .......... .......... .......... .......... .......... 50% 14.5M 3s 2024-08-25T19:47:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85200K .......... .......... .......... .......... .......... 50% 24.6M 3s 2024-08-25T19:47:51.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85250K .......... .......... .......... .......... .......... 50% 28.1M 3s 2024-08-25T19:47:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85300K .......... .......... .......... .......... .......... 50% 40.3M 3s 2024-08-25T19:47:51.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85350K .......... .......... .......... .......... .......... 50% 20.6M 3s 2024-08-25T19:47:51.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85400K .......... .......... .......... .......... .......... 50% 41.7M 3s 2024-08-25T19:47:51.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85450K .......... .......... .......... .......... .......... 50% 23.0M 3s 2024-08-25T19:47:51.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85500K .......... .......... .......... .......... .......... 50% 37.9M 3s 2024-08-25T19:47:51.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85550K .......... .......... .......... .......... .......... 50% 16.5M 3s 2024-08-25T19:47:51.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85600K .......... .......... .......... .......... .......... 50% 50.5M 3s 2024-08-25T19:47:51.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85650K .......... .......... .......... .......... .......... 50% 22.8M 3s 2024-08-25T19:47:51.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85700K .......... .......... .......... .......... .......... 50% 40.2M 3s 2024-08-25T19:47:51.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85750K .......... .......... .......... .......... .......... 50% 24.8M 3s 2024-08-25T19:47:51.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85800K .......... .......... .......... .......... .......... 50% 32.5M 3s 2024-08-25T19:47:51.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85850K .......... .......... .......... .......... .......... 50% 19.7M 3s 2024-08-25T19:47:51.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85900K .......... .......... .......... .......... .......... 50% 39.7M 3s 2024-08-25T19:47:51.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 85950K .......... .......... .......... .......... .......... 51% 20.2M 3s 2024-08-25T19:47:51.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86000K .......... .......... .......... .......... .......... 51% 24.1M 3s 2024-08-25T19:47:51.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86050K .......... .......... .......... .......... .......... 51% 38.0M 3s 2024-08-25T19:47:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86100K .......... .......... .......... .......... .......... 51% 22.8M 3s 2024-08-25T19:47:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86150K .......... .......... .......... .......... .......... 51% 14.6M 3s 2024-08-25T19:47:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86200K .......... .......... .......... .......... .......... 51% 30.0M 3s 2024-08-25T19:47:51.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86250K .......... .......... .......... .......... .......... 51% 28.0M 3s 2024-08-25T19:47:51.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86300K .......... .......... .......... .......... .......... 51% 33.9M 3s 2024-08-25T19:47:51.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86350K .......... .......... .......... .......... .......... 51% 17.6M 3s 2024-08-25T19:47:51.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86400K .......... .......... .......... .......... .......... 51% 20.4M 3s 2024-08-25T19:47:51.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86450K .......... .......... .......... .......... .......... 51% 56.9M 3s 2024-08-25T19:47:51.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86500K .......... .......... .......... .......... .......... 51% 21.8M 3s 2024-08-25T19:47:51.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86550K .......... .......... .......... .......... .......... 51% 38.2M 3s 2024-08-25T19:47:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86600K .......... .......... .......... .......... .......... 51% 25.3M 3s 2024-08-25T19:47:51.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86650K .......... .......... .......... .......... .......... 51% 16.2M 3s 2024-08-25T19:47:51.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86700K .......... .......... .......... .......... .......... 51% 71.5M 3s 2024-08-25T19:47:51.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86750K .......... .......... .......... .......... .......... 51% 21.5M 3s 2024-08-25T19:47:51.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86800K .......... .......... .......... .......... .......... 51% 26.4M 3s 2024-08-25T19:47:51.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86850K .......... .......... .......... .......... .......... 51% 40.3M 3s 2024-08-25T19:47:51.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86900K .......... .......... .......... .......... .......... 51% 23.8M 3s 2024-08-25T19:47:51.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 86950K .......... .......... .......... .......... .......... 51% 43.2M 3s 2024-08-25T19:47:51.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87000K .......... .......... .......... .......... .......... 51% 21.5M 3s 2024-08-25T19:47:51.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87050K .......... .......... .......... .......... .......... 51% 34.4M 3s 2024-08-25T19:47:51.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87100K .......... .......... .......... .......... .......... 51% 28.1M 3s 2024-08-25T19:47:51.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87150K .......... .......... .......... .......... .......... 51% 22.9M 3s 2024-08-25T19:47:51.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87200K .......... .......... .......... .......... .......... 51% 22.6M 3s 2024-08-25T19:47:51.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87250K .......... .......... .......... .......... .......... 51% 52.2M 3s 2024-08-25T19:47:51.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87300K .......... .......... .......... .......... .......... 51% 24.3M 3s 2024-08-25T19:47:51.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87350K .......... .......... .......... .......... .......... 51% 38.9M 3s 2024-08-25T19:47:51.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87400K .......... .......... .......... .......... .......... 51% 23.5M 3s 2024-08-25T19:47:51.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87450K .......... .......... .......... .......... .......... 51% 33.9M 3s 2024-08-25T19:47:51.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87500K .......... .......... .......... .......... .......... 51% 26.1M 3s 2024-08-25T19:47:51.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87550K .......... .......... .......... .......... .......... 51% 26.8M 3s 2024-08-25T19:47:51.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87600K .......... .......... .......... .......... .......... 51% 25.4M 3s 2024-08-25T19:47:51.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87650K .......... .......... .......... .......... .......... 52% 34.2M 3s 2024-08-25T19:47:51.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87700K .......... .......... .......... .......... .......... 52% 39.3M 3s 2024-08-25T19:47:51.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87750K .......... .......... .......... .......... .......... 52% 15.0M 3s 2024-08-25T19:47:51.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87800K .......... .......... .......... .......... .......... 52% 28.4M 3s 2024-08-25T19:47:51.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87850K .......... .......... .......... .......... .......... 52% 26.4M 3s 2024-08-25T19:47:51.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87900K .......... .......... .......... .......... .......... 52% 27.9M 3s 2024-08-25T19:47:51.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 87950K .......... .......... .......... .......... .......... 52% 24.2M 3s 2024-08-25T19:47:51.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88000K .......... .......... .......... .......... .......... 52% 33.8M 3s 2024-08-25T19:47:51.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88050K .......... .......... .......... .......... .......... 52% 27.0M 3s 2024-08-25T19:47:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88100K .......... .......... .......... .......... .......... 52% 43.1M 3s 2024-08-25T19:47:51.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88150K .......... .......... .......... .......... .......... 52% 15.4M 3s 2024-08-25T19:47:51.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88200K .......... .......... .......... .......... .......... 52% 43.5M 3s 2024-08-25T19:47:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88250K .......... .......... .......... .......... .......... 52% 27.3M 3s 2024-08-25T19:47:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88300K .......... .......... .......... .......... .......... 52% 39.3M 3s 2024-08-25T19:47:51.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88350K .......... .......... .......... .......... .......... 52% 16.1M 3s 2024-08-25T19:47:51.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88400K .......... .......... .......... .......... .......... 52% 41.1M 3s 2024-08-25T19:47:51.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88450K .......... .......... .......... .......... .......... 52% 25.4M 3s 2024-08-25T19:47:51.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88500K .......... .......... .......... .......... .......... 52% 37.9M 3s 2024-08-25T19:47:51.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88550K .......... .......... .......... .......... .......... 52% 15.9M 3s 2024-08-25T19:47:51.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88600K .......... .......... .......... .......... .......... 52% 39.5M 3s 2024-08-25T19:47:51.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88650K .......... .......... .......... .......... .......... 52% 26.3M 3s 2024-08-25T19:47:51.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88700K .......... .......... .......... .......... .......... 52% 42.0M 3s 2024-08-25T19:47:51.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88750K .......... .......... .......... .......... .......... 52% 13.7M 3s 2024-08-25T19:47:51.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88800K .......... .......... .......... .......... .......... 52% 24.7M 3s 2024-08-25T19:47:51.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88850K .......... .......... .......... .......... .......... 52% 36.9M 3s 2024-08-25T19:47:51.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88900K .......... .......... .......... .......... .......... 52% 21.6M 3s 2024-08-25T19:47:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 88950K .......... .......... .......... .......... .......... 52% 23.2M 3s 2024-08-25T19:47:51.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89000K .......... .......... .......... .......... .......... 52% 41.1M 3s 2024-08-25T19:47:51.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89050K .......... .......... .......... .......... .......... 52% 22.9M 3s 2024-08-25T19:47:51.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89100K .......... .......... .......... .......... .......... 52% 41.2M 3s 2024-08-25T19:47:51.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89150K .......... .......... .......... .......... .......... 52% 21.4M 3s 2024-08-25T19:47:51.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89200K .......... .......... .......... .......... .......... 52% 16.7M 3s 2024-08-25T19:47:51.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89250K .......... .......... .......... .......... .......... 52% 43.5M 3s 2024-08-25T19:47:51.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89300K .......... .......... .......... .......... .......... 53% 24.8M 3s 2024-08-25T19:47:51.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89350K .......... .......... .......... .......... .......... 53% 27.6M 3s 2024-08-25T19:47:51.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89400K .......... .......... .......... .......... .......... 53% 26.8M 3s 2024-08-25T19:47:51.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89450K .......... .......... .......... .......... .......... 53% 73.9M 3s 2024-08-25T19:47:51.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89500K .......... .......... .......... .......... .......... 53% 22.3M 3s 2024-08-25T19:47:51.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89550K .......... .......... .......... .......... .......... 53% 22.7M 3s 2024-08-25T19:47:51.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89600K .......... .......... .......... .......... .......... 53% 40.2M 3s 2024-08-25T19:47:51.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89650K .......... .......... .......... .......... .......... 53% 25.9M 3s 2024-08-25T19:47:51.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89700K .......... .......... .......... .......... .......... 53% 24.4M 3s 2024-08-25T19:47:51.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89750K .......... .......... .......... .......... .......... 53% 38.9M 3s 2024-08-25T19:47:51.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89800K .......... .......... .......... .......... .......... 53% 26.2M 3s 2024-08-25T19:47:51.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89850K .......... .......... .......... .......... .......... 53% 26.4M 3s 2024-08-25T19:47:51.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89900K .......... .......... .......... .......... .......... 53% 21.7M 3s 2024-08-25T19:47:51.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 89950K .......... .......... .......... .......... .......... 53% 21.1M 3s 2024-08-25T19:47:51.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90000K .......... .......... .......... .......... .......... 53% 26.4M 3s 2024-08-25T19:47:51.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90050K .......... .......... .......... .......... .......... 53% 16.3M 3s 2024-08-25T19:47:51.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90100K .......... .......... .......... .......... .......... 53% 27.5M 3s 2024-08-25T19:47:51.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90150K .......... .......... .......... .......... .......... 53% 39.1M 3s 2024-08-25T19:47:51.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90200K .......... .......... .......... .......... .......... 53% 34.3M 3s 2024-08-25T19:47:51.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90250K .......... .......... .......... .......... .......... 53% 18.8M 3s 2024-08-25T19:47:51.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90300K .......... .......... .......... .......... .......... 53% 23.8M 3s 2024-08-25T19:47:51.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90350K .......... .......... .......... .......... .......... 53% 24.8M 3s 2024-08-25T19:47:51.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90400K .......... .......... .......... .......... .......... 53% 38.9M 3s 2024-08-25T19:47:51.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90450K .......... .......... .......... .......... .......... 53% 16.0M 3s 2024-08-25T19:47:51.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90500K .......... .......... .......... .......... .......... 53% 38.3M 3s 2024-08-25T19:47:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90550K .......... .......... .......... .......... .......... 53% 25.3M 3s 2024-08-25T19:47:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90600K .......... .......... .......... .......... .......... 53% 34.5M 3s 2024-08-25T19:47:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90650K .......... .......... .......... .......... .......... 53% 25.6M 3s 2024-08-25T19:47:51.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90700K .......... .......... .......... .......... .......... 53% 22.7M 3s 2024-08-25T19:47:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90750K .......... .......... .......... .......... .......... 53% 22.4M 3s 2024-08-25T19:47:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90800K .......... .......... .......... .......... .......... 53% 39.9M 3s 2024-08-25T19:47:51.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90850K .......... .......... .......... .......... .......... 53% 30.1M 3s 2024-08-25T19:47:51.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90900K .......... .......... .......... .......... .......... 53% 26.1M 3s 2024-08-25T19:47:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 90950K .......... .......... .......... .......... .......... 53% 28.9M 3s 2024-08-25T19:47:51.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91000K .......... .......... .......... .......... .......... 54% 19.5M 3s 2024-08-25T19:47:51.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91050K .......... .......... .......... .......... .......... 54% 22.8M 3s 2024-08-25T19:47:51.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91100K .......... .......... .......... .......... .......... 54% 50.8M 3s 2024-08-25T19:47:51.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91150K .......... .......... .......... .......... .......... 54% 17.5M 3s 2024-08-25T19:47:51.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91200K .......... .......... .......... .......... .......... 54% 22.5M 3s 2024-08-25T19:47:51.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91250K .......... .......... .......... .......... .......... 54% 23.1M 3s 2024-08-25T19:47:51.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91300K .......... .......... .......... .......... .......... 54% 40.4M 3s 2024-08-25T19:47:51.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91350K .......... .......... .......... .......... .......... 54% 27.7M 3s 2024-08-25T19:47:51.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91400K .......... .......... .......... .......... .......... 54% 36.0M 3s 2024-08-25T19:47:51.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91450K .......... .......... .......... .......... .......... 54% 26.9M 3s 2024-08-25T19:47:51.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91500K .......... .......... .......... .......... .......... 54% 39.2M 3s 2024-08-25T19:47:51.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91550K .......... .......... .......... .......... .......... 54% 24.8M 3s 2024-08-25T19:47:51.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91600K .......... .......... .......... .......... .......... 54% 25.6M 3s 2024-08-25T19:47:51.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91650K .......... .......... .......... .......... .......... 54% 41.6M 3s 2024-08-25T19:47:51.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91700K .......... .......... .......... .......... .......... 54% 23.8M 3s 2024-08-25T19:47:51.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91750K .......... .......... .......... .......... .......... 54% 28.5M 3s 2024-08-25T19:47:51.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91800K .......... .......... .......... .......... .......... 54% 40.0M 3s 2024-08-25T19:47:51.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91850K .......... .......... .......... .......... .......... 54% 20.6M 3s 2024-08-25T19:47:51.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91900K .......... .......... .......... .......... .......... 54% 27.4M 3s 2024-08-25T19:47:51.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 91950K .......... .......... .......... .......... .......... 54% 26.6M 3s 2024-08-25T19:47:51.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92000K .......... .......... .......... .......... .......... 54% 24.6M 3s 2024-08-25T19:47:51.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92050K .......... .......... .......... .......... .......... 54% 35.8M 3s 2024-08-25T19:47:51.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92100K .......... .......... .......... .......... .......... 54% 22.4M 3s 2024-08-25T19:47:51.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92150K .......... .......... .......... .......... .......... 54% 36.2M 3s 2024-08-25T19:47:51.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92200K .......... .......... .......... .......... .......... 54% 23.7M 3s 2024-08-25T19:47:51.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92250K .......... .......... .......... .......... .......... 54% 31.5M 3s 2024-08-25T19:47:51.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92300K .......... .......... .......... .......... .......... 54% 23.9M 3s 2024-08-25T19:47:51.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92350K .......... .......... .......... .......... .......... 54% 25.0M 3s 2024-08-25T19:47:51.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92400K .......... .......... .......... .......... .......... 54% 22.1M 3s 2024-08-25T19:47:51.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92450K .......... .......... .......... .......... .......... 54% 39.2M 3s 2024-08-25T19:47:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92500K .......... .......... .......... .......... .......... 54% 25.6M 3s 2024-08-25T19:47:51.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92550K .......... .......... .......... .......... .......... 54% 35.6M 3s 2024-08-25T19:47:51.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92600K .......... .......... .......... .......... .......... 54% 25.0M 3s 2024-08-25T19:47:51.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92650K .......... .......... .......... .......... .......... 54% 14.6M 3s 2024-08-25T19:47:51.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92700K .......... .......... .......... .......... .......... 55% 35.2M 3s 2024-08-25T19:47:51.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92750K .......... .......... .......... .......... .......... 55% 22.2M 3s 2024-08-25T19:47:51.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92800K .......... .......... .......... .......... .......... 55% 46.0M 3s 2024-08-25T19:47:51.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92850K .......... .......... .......... .......... .......... 55% 19.0M 3s 2024-08-25T19:47:51.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92900K .......... .......... .......... .......... .......... 55% 23.0M 3s 2024-08-25T19:47:51.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 92950K .......... .......... .......... .......... .......... 55% 35.6M 3s 2024-08-25T19:47:51.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93000K .......... .......... .......... .......... .......... 55% 28.9M 3s 2024-08-25T19:47:51.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93050K .......... .......... .......... .......... .......... 55% 43.0M 3s 2024-08-25T19:47:51.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93100K .......... .......... .......... .......... .......... 55% 21.5M 3s 2024-08-25T19:47:51.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93150K .......... .......... .......... .......... .......... 55% 9.69M 3s 2024-08-25T19:47:51.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93200K .......... .......... .......... .......... .......... 55% 40.4M 3s 2024-08-25T19:47:51.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93250K .......... .......... .......... .......... .......... 55% 169M 3s 2024-08-25T19:47:51.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93300K .......... .......... .......... .......... .......... 55% 165M 3s 2024-08-25T19:47:51.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93350K .......... .......... .......... .......... .......... 55% 13.5M 3s 2024-08-25T19:47:51.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93400K .......... .......... .......... .......... .......... 55% 49.8M 3s 2024-08-25T19:47:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93450K .......... .......... .......... .......... .......... 55% 21.3M 3s 2024-08-25T19:47:51.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93500K .......... .......... .......... .......... .......... 55% 22.2M 3s 2024-08-25T19:47:51.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93550K .......... .......... .......... .......... .......... 55% 24.0M 3s 2024-08-25T19:47:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93600K .......... .......... .......... .......... .......... 55% 40.6M 3s 2024-08-25T19:47:51.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93650K .......... .......... .......... .......... .......... 55% 21.2M 3s 2024-08-25T19:47:51.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93700K .......... .......... .......... .......... .......... 55% 46.1M 3s 2024-08-25T19:47:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93750K .......... .......... .......... .......... .......... 55% 22.9M 3s 2024-08-25T19:47:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93800K .......... .......... .......... .......... .......... 55% 39.8M 3s 2024-08-25T19:47:51.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93850K .......... .......... .......... .......... .......... 55% 26.1M 3s 2024-08-25T19:47:51.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93900K .......... .......... .......... .......... .......... 55% 33.6M 3s 2024-08-25T19:47:51.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 93950K .......... .......... .......... .......... .......... 55% 18.0M 3s 2024-08-25T19:47:51.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94000K .......... .......... .......... .......... .......... 55% 24.8M 3s 2024-08-25T19:47:51.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94050K .......... .......... .......... .......... .......... 55% 21.6M 3s 2024-08-25T19:47:51.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94100K .......... .......... .......... .......... .......... 55% 48.5M 3s 2024-08-25T19:47:51.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94150K .......... .......... .......... .......... .......... 55% 22.8M 3s 2024-08-25T19:47:51.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94200K .......... .......... .......... .......... .......... 55% 41.9M 3s 2024-08-25T19:47:51.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94250K .......... .......... .......... .......... .......... 55% 23.6M 3s 2024-08-25T19:47:51.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94300K .......... .......... .......... .......... .......... 55% 46.5M 3s 2024-08-25T19:47:51.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94350K .......... .......... .......... .......... .......... 56% 21.2M 3s 2024-08-25T19:47:51.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94400K .......... .......... .......... .......... .......... 56% 26.2M 3s 2024-08-25T19:47:51.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94450K .......... .......... .......... .......... .......... 56% 26.8M 3s 2024-08-25T19:47:51.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94500K .......... .......... .......... .......... .......... 56% 11.8M 3s 2024-08-25T19:47:51.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94550K .......... .......... .......... .......... .......... 56% 47.1M 3s 2024-08-25T19:47:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94600K .......... .......... .......... .......... .......... 56% 24.7M 3s 2024-08-25T19:47:51.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94650K .......... .......... .......... .......... .......... 56% 22.5M 3s 2024-08-25T19:47:51.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94700K .......... .......... .......... .......... .......... 56% 26.8M 3s 2024-08-25T19:47:51.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94750K .......... .......... .......... .......... .......... 56% 22.3M 3s 2024-08-25T19:47:51.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94800K .......... .......... .......... .......... .......... 56% 27.8M 3s 2024-08-25T19:47:51.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94850K .......... .......... .......... .......... .......... 56% 37.7M 3s 2024-08-25T19:47:51.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94900K .......... .......... .......... .......... .......... 56% 25.1M 3s 2024-08-25T19:47:51.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 94950K .......... .......... .......... .......... .......... 56% 36.7M 3s 2024-08-25T19:47:51.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95000K .......... .......... .......... .......... .......... 56% 23.2M 3s 2024-08-25T19:47:51.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95050K .......... .......... .......... .......... .......... 56% 48.4M 3s 2024-08-25T19:47:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95100K .......... .......... .......... .......... .......... 56% 23.7M 3s 2024-08-25T19:47:51.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95150K .......... .......... .......... .......... .......... 56% 22.9M 3s 2024-08-25T19:47:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95200K .......... .......... .......... .......... .......... 56% 22.7M 3s 2024-08-25T19:47:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95250K .......... .......... .......... .......... .......... 56% 44.2M 3s 2024-08-25T19:47:51.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95300K .......... .......... .......... .......... .......... 56% 24.3M 3s 2024-08-25T19:47:51.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95350K .......... .......... .......... .......... .......... 56% 44.4M 3s 2024-08-25T19:47:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95400K .......... .......... .......... .......... .......... 56% 20.7M 3s 2024-08-25T19:47:52.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95450K .......... .......... .......... .......... .......... 56% 20.1M 3s 2024-08-25T19:47:52.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95500K .......... .......... .......... .......... .......... 56% 22.8M 3s 2024-08-25T19:47:52.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95550K .......... .......... .......... .......... .......... 56% 24.7M 3s 2024-08-25T19:47:52.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95600K .......... .......... .......... .......... .......... 56% 45.6M 3s 2024-08-25T19:47:52.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95650K .......... .......... .......... .......... .......... 56% 35.1M 3s 2024-08-25T19:47:52.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95700K .......... .......... .......... .......... .......... 56% 17.9M 3s 2024-08-25T19:47:52.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95750K .......... .......... .......... .......... .......... 56% 41.9M 3s 2024-08-25T19:47:52.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95800K .......... .......... .......... .......... .......... 56% 24.0M 3s 2024-08-25T19:47:52.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95850K .......... .......... .......... .......... .......... 56% 37.5M 3s 2024-08-25T19:47:52.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95900K .......... .......... .......... .......... .......... 56% 35.3M 3s 2024-08-25T19:47:52.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 95950K .......... .......... .......... .......... .......... 56% 18.8M 3s 2024-08-25T19:47:52.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96000K .......... .......... .......... .......... .......... 56% 39.9M 3s 2024-08-25T19:47:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96050K .......... .......... .......... .......... .......... 57% 23.9M 3s 2024-08-25T19:47:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96100K .......... .......... .......... .......... .......... 57% 40.3M 3s 2024-08-25T19:47:52.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96150K .......... .......... .......... .......... .......... 57% 23.3M 3s 2024-08-25T19:47:52.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96200K .......... .......... .......... .......... .......... 57% 46.1M 3s 2024-08-25T19:47:52.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96250K .......... .......... .......... .......... .......... 57% 15.8M 3s 2024-08-25T19:47:52.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96300K .......... .......... .......... .......... .......... 57% 25.0M 3s 2024-08-25T19:47:52.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96350K .......... .......... .......... .......... .......... 57% 24.5M 3s 2024-08-25T19:47:52.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96400K .......... .......... .......... .......... .......... 57% 37.2M 3s 2024-08-25T19:47:52.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96450K .......... .......... .......... .......... .......... 57% 27.1M 3s 2024-08-25T19:47:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96500K .......... .......... .......... .......... .......... 57% 42.0M 3s 2024-08-25T19:47:52.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96550K .......... .......... .......... .......... .......... 57% 22.8M 3s 2024-08-25T19:47:52.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96600K .......... .......... .......... .......... .......... 57% 49.9M 3s 2024-08-25T19:47:52.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96650K .......... .......... .......... .......... .......... 57% 25.3M 3s 2024-08-25T19:47:52.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96700K .......... .......... .......... .......... .......... 57% 33.1M 3s 2024-08-25T19:47:52.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96750K .......... .......... .......... .......... .......... 57% 19.8M 3s 2024-08-25T19:47:52.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96800K .......... .......... .......... .......... .......... 57% 30.1M 3s 2024-08-25T19:47:52.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96850K .......... .......... .......... .......... .......... 57% 22.5M 3s 2024-08-25T19:47:52.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96900K .......... .......... .......... .......... .......... 57% 48.4M 3s 2024-08-25T19:47:52.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 96950K .......... .......... .......... .......... .......... 57% 23.0M 3s 2024-08-25T19:47:52.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97000K .......... .......... .......... .......... .......... 57% 45.9M 3s 2024-08-25T19:47:52.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97050K .......... .......... .......... .......... .......... 57% 22.1M 3s 2024-08-25T19:47:52.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97100K .......... .......... .......... .......... .......... 57% 50.3M 3s 2024-08-25T19:47:52.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97150K .......... .......... .......... .......... .......... 57% 22.7M 3s 2024-08-25T19:47:52.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97200K .......... .......... .......... .......... .......... 57% 24.5M 3s 2024-08-25T19:47:52.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97250K .......... .......... .......... .......... .......... 57% 42.1M 3s 2024-08-25T19:47:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97300K .......... .......... .......... .......... .......... 57% 23.3M 3s 2024-08-25T19:47:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97350K .......... .......... .......... .......... .......... 57% 45.2M 3s 2024-08-25T19:47:52.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97400K .......... .......... .......... .......... .......... 57% 22.2M 3s 2024-08-25T19:47:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97450K .......... .......... .......... .......... .......... 57% 21.9M 3s 2024-08-25T19:47:52.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97500K .......... .......... .......... .......... .......... 57% 44.6M 3s 2024-08-25T19:47:52.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97550K .......... .......... .......... .......... .......... 57% 23.2M 3s 2024-08-25T19:47:52.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97600K .......... .......... .......... .......... .......... 57% 27.4M 3s 2024-08-25T19:47:52.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97650K .......... .......... .......... .......... .......... 57% 44.2M 3s 2024-08-25T19:47:52.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97700K .......... .......... .......... .......... .......... 57% 25.3M 3s 2024-08-25T19:47:52.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97750K .......... .......... .......... .......... .......... 58% 33.8M 3s 2024-08-25T19:47:52.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97800K .......... .......... .......... .......... .......... 58% 22.4M 3s 2024-08-25T19:47:52.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97850K .......... .......... .......... .......... .......... 58% 39.5M 3s 2024-08-25T19:47:52.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97900K .......... .......... .......... .......... .......... 58% 25.7M 3s 2024-08-25T19:47:52.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 97950K .......... .......... .......... .......... .......... 58% 24.2M 3s 2024-08-25T19:47:52.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98000K .......... .......... .......... .......... .......... 58% 32.0M 3s 2024-08-25T19:47:52.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98050K .......... .......... .......... .......... .......... 58% 26.2M 3s 2024-08-25T19:47:52.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98100K .......... .......... .......... .......... .......... 58% 25.4M 3s 2024-08-25T19:47:52.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98150K .......... .......... .......... .......... .......... 58% 34.4M 3s 2024-08-25T19:47:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98200K .......... .......... .......... .......... .......... 58% 41.5M 3s 2024-08-25T19:47:52.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98250K .......... .......... .......... .......... .......... 58% 25.5M 3s 2024-08-25T19:47:52.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98300K .......... .......... .......... .......... .......... 58% 27.2M 3s 2024-08-25T19:47:52.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98350K .......... .......... .......... .......... .......... 58% 20.0M 3s 2024-08-25T19:47:52.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98400K .......... .......... .......... .......... .......... 58% 41.5M 3s 2024-08-25T19:47:52.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98450K .......... .......... .......... .......... .......... 58% 23.4M 3s 2024-08-25T19:47:52.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98500K .......... .......... .......... .......... .......... 58% 33.2M 3s 2024-08-25T19:47:52.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98550K .......... .......... .......... .......... .......... 58% 25.0M 3s 2024-08-25T19:47:52.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98600K .......... .......... .......... .......... .......... 58% 22.9M 3s 2024-08-25T19:47:52.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98650K .......... .......... .......... .......... .......... 58% 43.0M 3s 2024-08-25T19:47:52.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98700K .......... .......... .......... .......... .......... 58% 21.1M 3s 2024-08-25T19:47:52.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98750K .......... .......... .......... .......... .......... 58% 19.3M 3s 2024-08-25T19:47:52.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98800K .......... .......... .......... .......... .......... 58% 46.9M 3s 2024-08-25T19:47:52.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98850K .......... .......... .......... .......... .......... 58% 22.8M 3s 2024-08-25T19:47:52.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98900K .......... .......... .......... .......... .......... 58% 46.7M 3s 2024-08-25T19:47:52.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 98950K .......... .......... .......... .......... .......... 58% 24.9M 3s 2024-08-25T19:47:52.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99000K .......... .......... .......... .......... .......... 58% 39.4M 3s 2024-08-25T19:47:52.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99050K .......... .......... .......... .......... .......... 58% 25.0M 3s 2024-08-25T19:47:52.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99100K .......... .......... .......... .......... .......... 58% 36.6M 3s 2024-08-25T19:47:52.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99150K .......... .......... .......... .......... .......... 58% 16.9M 3s 2024-08-25T19:47:52.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99200K .......... .......... .......... .......... .......... 58% 42.4M 3s 2024-08-25T19:47:52.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99250K .......... .......... .......... .......... .......... 58% 23.9M 3s 2024-08-25T19:47:52.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99300K .......... .......... .......... .......... .......... 58% 42.0M 3s 2024-08-25T19:47:52.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99350K .......... .......... .......... .......... .......... 58% 26.7M 3s 2024-08-25T19:47:52.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99400K .......... .......... .......... .......... .......... 58% 31.4M 3s 2024-08-25T19:47:52.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99450K .......... .......... .......... .......... .......... 59% 23.7M 3s 2024-08-25T19:47:52.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99500K .......... .......... .......... .......... .......... 59% 41.7M 3s 2024-08-25T19:47:52.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99550K .......... .......... .......... .......... .......... 59% 21.2M 3s 2024-08-25T19:47:52.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99600K .......... .......... .......... .......... .......... 59% 26.2M 3s 2024-08-25T19:47:52.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99650K .......... .......... .......... .......... .......... 59% 49.0M 3s 2024-08-25T19:47:52.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99700K .......... .......... .......... .......... .......... 59% 25.2M 3s 2024-08-25T19:47:52.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99750K .......... .......... .......... .......... .......... 59% 24.4M 3s 2024-08-25T19:47:52.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99800K .......... .......... .......... .......... .......... 59% 24.8M 3s 2024-08-25T19:47:52.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99850K .......... .......... .......... .......... .......... 59% 23.5M 3s 2024-08-25T19:47:52.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99900K .......... .......... .......... .......... .......... 59% 39.2M 3s 2024-08-25T19:47:52.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 99950K .......... .......... .......... .......... .......... 59% 20.0M 3s 2024-08-25T19:47:52.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100000K .......... .......... .......... .......... .......... 59% 10.2M 3s 2024-08-25T19:47:52.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100050K .......... .......... .......... .......... .......... 59% 42.9M 3s 2024-08-25T19:47:52.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100100K .......... .......... .......... .......... .......... 59% 21.2M 3s 2024-08-25T19:47:52.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100150K .......... .......... .......... .......... .......... 59% 43.6M 3s 2024-08-25T19:47:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100200K .......... .......... .......... .......... .......... 59% 23.9M 3s 2024-08-25T19:47:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100250K .......... .......... .......... .......... .......... 59% 37.7M 3s 2024-08-25T19:47:52.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100300K .......... .......... .......... .......... .......... 59% 23.5M 3s 2024-08-25T19:47:52.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100350K .......... .......... .......... .......... .......... 59% 17.4M 3s 2024-08-25T19:47:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100400K .......... .......... .......... .......... .......... 59% 29.0M 3s 2024-08-25T19:47:52.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100450K .......... .......... .......... .......... .......... 59% 13.4M 3s 2024-08-25T19:47:52.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100500K .......... .......... .......... .......... .......... 59% 11.8M 3s 2024-08-25T19:47:52.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100550K .......... .......... .......... .......... .......... 59% 35.5M 3s 2024-08-25T19:47:52.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100600K .......... .......... .......... .......... .......... 59% 26.7M 3s 2024-08-25T19:47:52.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100650K .......... .......... .......... .......... .......... 59% 36.2M 3s 2024-08-25T19:47:52.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100700K .......... .......... .......... .......... .......... 59% 13.6M 3s 2024-08-25T19:47:52.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100750K .......... .......... .......... .......... .......... 59% 25.5M 3s 2024-08-25T19:47:52.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100800K .......... .......... .......... .......... .......... 59% 25.1M 3s 2024-08-25T19:47:52.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100850K .......... .......... .......... .......... .......... 59% 19.8M 3s 2024-08-25T19:47:52.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100900K .......... .......... .......... .......... .......... 59% 671K 3s 2024-08-25T19:47:52.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:100950K .......... .......... .......... .......... .......... 59% 73.6M 3s 2024-08-25T19:47:52.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101000K .......... .......... .......... .......... .......... 59% 114M 3s 2024-08-25T19:47:52.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101050K .......... .......... .......... .......... .......... 59% 84.1M 3s 2024-08-25T19:47:52.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101100K .......... .......... .......... .......... .......... 60% 101M 3s 2024-08-25T19:47:52.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101150K .......... .......... .......... .......... .......... 60% 37.9M 3s 2024-08-25T19:47:52.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101200K .......... .......... .......... .......... .......... 60% 53.1M 3s 2024-08-25T19:47:52.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101250K .......... .......... .......... .......... .......... 60% 117M 3s 2024-08-25T19:47:52.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101300K .......... .......... .......... .......... .......... 60% 106M 3s 2024-08-25T19:47:52.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101350K .......... .......... .......... .......... .......... 60% 103M 3s 2024-08-25T19:47:52.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101400K .......... .......... .......... .......... .......... 60% 105M 3s 2024-08-25T19:47:52.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101450K .......... .......... .......... .......... .......... 60% 117M 3s 2024-08-25T19:47:52.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101500K .......... .......... .......... .......... .......... 60% 114M 3s 2024-08-25T19:47:52.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101550K .......... .......... .......... .......... .......... 60% 89.5M 3s 2024-08-25T19:47:52.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101600K .......... .......... .......... .......... .......... 60% 115M 3s 2024-08-25T19:47:52.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101650K .......... .......... .......... .......... .......... 60% 7.99M 3s 2024-08-25T19:47:52.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101700K .......... .......... .......... .......... .......... 60% 115M 3s 2024-08-25T19:47:52.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101750K .......... .......... .......... .......... .......... 60% 102M 3s 2024-08-25T19:47:52.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101800K .......... .......... .......... .......... .......... 60% 99.1M 3s 2024-08-25T19:47:52.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101850K .......... .......... .......... .......... .......... 60% 112M 3s 2024-08-25T19:47:52.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101900K .......... .......... .......... .......... .......... 60% 105M 3s 2024-08-25T19:47:52.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:101950K .......... .......... .......... .......... .......... 60% 102M 3s 2024-08-25T19:47:52.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102000K .......... .......... .......... .......... .......... 60% 103M 3s 2024-08-25T19:47:52.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102050K .......... .......... .......... .......... .......... 60% 112M 3s 2024-08-25T19:47:52.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102100K .......... .......... .......... .......... .......... 60% 108M 3s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102150K .......... .......... .......... .......... .......... 60% 100M 3s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102200K .......... .......... .......... .......... .......... 60% 99.4M 3s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102250K .......... .......... .......... .......... .......... 60% 112M 2s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102300K .......... .......... .......... .......... .......... 60% 100M 2s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102350K .......... .......... .......... .......... .......... 60% 94.9M 2s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102400K .......... .......... .......... .......... .......... 60% 112M 2s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102450K .......... .......... .......... .......... .......... 60% 107M 2s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102500K .......... .......... .......... .......... .......... 60% 113M 2s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102550K .......... .......... .......... .......... .......... 60% 95.4M 2s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102600K .......... .......... .......... .......... .......... 60% 116M 2s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102650K .......... .......... .......... .......... .......... 60% 104M 2s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102700K .......... .......... .......... .......... .......... 60% 115M 2s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102750K .......... .......... .......... .......... .......... 60% 86.7M 2s 2024-08-25T19:47:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102800K .......... .......... .......... .......... .......... 61% 112M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102850K .......... .......... .......... .......... .......... 61% 104M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102900K .......... .......... .......... .......... .......... 61% 112M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:102950K .......... .......... .......... .......... .......... 61% 98.3M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103000K .......... .......... .......... .......... .......... 61% 108M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103050K .......... .......... .......... .......... .......... 61% 113M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103100K .......... .......... .......... .......... .......... 61% 111M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103150K .......... .......... .......... .......... .......... 61% 91.0M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103200K .......... .......... .......... .......... .......... 61% 108M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103250K .......... .......... .......... .......... .......... 61% 110M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103300K .......... .......... .......... .......... .......... 61% 102M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103350K .......... .......... .......... .......... .......... 61% 109M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103400K .......... .......... .......... .......... .......... 61% 103M 2s 2024-08-25T19:47:52.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103450K .......... .......... .......... .......... .......... 61% 111M 2s 2024-08-25T19:47:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103500K .......... .......... .......... .......... .......... 61% 99.8M 2s 2024-08-25T19:47:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103550K .......... .......... .......... .......... .......... 61% 96.7M 2s 2024-08-25T19:47:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103600K .......... .......... .......... .......... .......... 61% 108M 2s 2024-08-25T19:47:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103650K .......... .......... .......... .......... .......... 61% 110M 2s 2024-08-25T19:47:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103700K .......... .......... .......... .......... .......... 61% 110M 2s 2024-08-25T19:47:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103750K .......... .......... .......... .......... .......... 61% 99.8M 2s 2024-08-25T19:47:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103800K .......... .......... .......... .......... .......... 61% 114M 2s 2024-08-25T19:47:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103850K .......... .......... .......... .......... .......... 61% 52.1M 2s 2024-08-25T19:47:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103900K .......... .......... .......... .......... .......... 61% 98.1M 2s 2024-08-25T19:47:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:103950K .......... .......... .......... .......... .......... 61% 23.6M 2s 2024-08-25T19:47:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104000K .......... .......... .......... .......... .......... 61% 26.1M 2s 2024-08-25T19:47:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104050K .......... .......... .......... .......... .......... 61% 27.9M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104100K .......... .......... .......... .......... .......... 61% 26.1M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104150K .......... .......... .......... .......... .......... 61% 25.3M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104200K .......... .......... .......... .......... .......... 61% 46.7M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104250K .......... .......... .......... .......... .......... 61% 36.1M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104300K .......... .......... .......... .......... .......... 61% 26.9M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104350K .......... .......... .......... .......... .......... 61% 24.6M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104400K .......... .......... .......... .......... .......... 61% 23.4M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104450K .......... .......... .......... .......... .......... 61% 50.1M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104500K .......... .......... .......... .......... .......... 62% 28.4M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104550K .......... .......... .......... .......... .......... 62% 31.7M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104600K .......... .......... .......... .......... .......... 62% 20.6M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104650K .......... .......... .......... .......... .......... 62% 36.2M 2s 2024-08-25T19:47:52.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104700K .......... .......... .......... .......... .......... 62% 36.9M 2s 2024-08-25T19:47:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104750K .......... .......... .......... .......... .......... 62% 23.2M 2s 2024-08-25T19:47:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104800K .......... .......... .......... .......... .......... 62% 19.6M 2s 2024-08-25T19:47:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104850K .......... .......... .......... .......... .......... 62% 40.9M 2s 2024-08-25T19:47:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104900K .......... .......... .......... .......... .......... 62% 23.2M 2s 2024-08-25T19:47:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:104950K .......... .......... .......... .......... .......... 62% 22.5M 2s 2024-08-25T19:47:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105000K .......... .......... .......... .......... .......... 62% 36.6M 2s 2024-08-25T19:47:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105050K .......... .......... .......... .......... .......... 62% 23.5M 2s 2024-08-25T19:47:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105100K .......... .......... .......... .......... .......... 62% 61.6M 2s 2024-08-25T19:47:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105150K .......... .......... .......... .......... .......... 62% 23.4M 2s 2024-08-25T19:47:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105200K .......... .......... .......... .......... .......... 62% 24.2M 2s 2024-08-25T19:47:52.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105250K .......... .......... .......... .......... .......... 62% 41.0M 2s 2024-08-25T19:47:52.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105300K .......... .......... .......... .......... .......... 62% 22.1M 2s 2024-08-25T19:47:52.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105350K .......... .......... .......... .......... .......... 62% 56.1M 2s 2024-08-25T19:47:52.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105400K .......... .......... .......... .......... .......... 62% 23.8M 2s 2024-08-25T19:47:52.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105450K .......... .......... .......... .......... .......... 62% 33.2M 2s 2024-08-25T19:47:52.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105500K .......... .......... .......... .......... .......... 62% 24.9M 2s 2024-08-25T19:47:52.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105550K .......... .......... .......... .......... .......... 62% 21.0M 2s 2024-08-25T19:47:52.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105600K .......... .......... .......... .......... .......... 62% 25.9M 2s 2024-08-25T19:47:52.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105650K .......... .......... .......... .......... .......... 62% 36.2M 2s 2024-08-25T19:47:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105700K .......... .......... .......... .......... .......... 62% 21.5M 2s 2024-08-25T19:47:52.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105750K .......... .......... .......... .......... .......... 62% 37.5M 2s 2024-08-25T19:47:52.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105800K .......... .......... .......... .......... .......... 62% 23.6M 2s 2024-08-25T19:47:52.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105850K .......... .......... .......... .......... .......... 62% 46.4M 2s 2024-08-25T19:47:52.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105900K .......... .......... .......... .......... .......... 62% 24.2M 2s 2024-08-25T19:47:52.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:105950K .......... .......... .......... .......... .......... 62% 24.8M 2s 2024-08-25T19:47:52.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106000K .......... .......... .......... .......... .......... 62% 31.6M 2s 2024-08-25T19:47:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106050K .......... .......... .......... .......... .......... 62% 27.9M 2s 2024-08-25T19:47:52.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106100K .......... .......... .......... .......... .......... 62% 22.4M 2s 2024-08-25T19:47:52.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106150K .......... .......... .......... .......... .......... 63% 38.8M 2s 2024-08-25T19:47:52.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106200K .......... .......... .......... .......... .......... 63% 35.5M 2s 2024-08-25T19:47:52.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106250K .......... .......... .......... .......... .......... 63% 29.2M 2s 2024-08-25T19:47:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106300K .......... .......... .......... .......... .......... 63% 22.8M 2s 2024-08-25T19:47:52.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106350K .......... .......... .......... .......... .......... 63% 23.4M 2s 2024-08-25T19:47:52.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106400K .......... .......... .......... .......... .......... 63% 47.4M 2s 2024-08-25T19:47:52.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106450K .......... .......... .......... .......... .......... 63% 23.3M 2s 2024-08-25T19:47:52.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106500K .......... .......... .......... .......... .......... 63% 36.6M 2s 2024-08-25T19:47:52.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106550K .......... .......... .......... .......... .......... 63% 24.3M 2s 2024-08-25T19:47:52.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106600K .......... .......... .......... .......... .......... 63% 46.0M 2s 2024-08-25T19:47:52.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106650K .......... .......... .......... .......... .......... 63% 24.2M 2s 2024-08-25T19:47:52.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106700K .......... .......... .......... .......... .......... 63% 36.6M 2s 2024-08-25T19:47:52.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106750K .......... .......... .......... .......... .......... 63% 17.5M 2s 2024-08-25T19:47:52.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106800K .......... .......... .......... .......... .......... 63% 38.5M 2s 2024-08-25T19:47:52.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106850K .......... .......... .......... .......... .......... 63% 23.5M 2s 2024-08-25T19:47:52.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106900K .......... .......... .......... .......... .......... 63% 46.2M 2s 2024-08-25T19:47:52.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:106950K .......... .......... .......... .......... .......... 63% 24.5M 2s 2024-08-25T19:47:52.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107000K .......... .......... .......... .......... .......... 63% 39.4M 2s 2024-08-25T19:47:52.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107050K .......... .......... .......... .......... .......... 63% 23.1M 2s 2024-08-25T19:47:52.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107100K .......... .......... .......... .......... .......... 63% 49.9M 2s 2024-08-25T19:47:52.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107150K .......... .......... .......... .......... .......... 63% 21.5M 2s 2024-08-25T19:47:52.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107200K .......... .......... .......... .......... .......... 63% 28.0M 2s 2024-08-25T19:47:52.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107250K .......... .......... .......... .......... .......... 63% 35.3M 2s 2024-08-25T19:47:52.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107300K .......... .......... .......... .......... .......... 63% 22.4M 2s 2024-08-25T19:47:52.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107350K .......... .......... .......... .......... .......... 63% 21.9M 2s 2024-08-25T19:47:52.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107400K .......... .......... .......... .......... .......... 63% 56.8M 2s 2024-08-25T19:47:52.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107450K .......... .......... .......... .......... .......... 63% 24.3M 2s 2024-08-25T19:47:52.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107500K .......... .......... .......... .......... .......... 63% 38.0M 2s 2024-08-25T19:47:52.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107550K .......... .......... .......... .......... .......... 63% 24.3M 2s 2024-08-25T19:47:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107600K .......... .......... .......... .......... .......... 63% 25.2M 2s 2024-08-25T19:47:52.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107650K .......... .......... .......... .......... .......... 63% 45.4M 2s 2024-08-25T19:47:52.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107700K .......... .......... .......... .......... .......... 63% 23.2M 2s 2024-08-25T19:47:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107750K .......... .......... .......... .......... .......... 63% 31.8M 2s 2024-08-25T19:47:52.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107800K .......... .......... .......... .......... .......... 63% 18.7M 2s 2024-08-25T19:47:52.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107850K .......... .......... .......... .......... .......... 64% 22.1M 2s 2024-08-25T19:47:52.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107900K .......... .......... .......... .......... .......... 64% 55.4M 2s 2024-08-25T19:47:52.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:107950K .......... .......... .......... .......... .......... 64% 9.31M 2s 2024-08-25T19:47:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108000K .......... .......... .......... .......... .......... 64% 20.7M 2s 2024-08-25T19:47:52.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108050K .......... .......... .......... .......... .......... 64% 38.8M 2s 2024-08-25T19:47:52.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108100K .......... .......... .......... .......... .......... 64% 22.9M 2s 2024-08-25T19:47:52.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108150K .......... .......... .......... .......... .......... 64% 51.7M 2s 2024-08-25T19:47:52.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108200K .......... .......... .......... .......... .......... 64% 17.0M 2s 2024-08-25T19:47:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108250K .......... .......... .......... .......... .......... 64% 11.9M 2s 2024-08-25T19:47:52.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108300K .......... .......... .......... .......... .......... 64% 26.2M 2s 2024-08-25T19:47:52.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108350K .......... .......... .......... .......... .......... 64% 5.16M 2s 2024-08-25T19:47:52.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108400K .......... .......... .......... .......... .......... 64% 17.1M 2s 2024-08-25T19:47:52.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108450K .......... .......... .......... .......... .......... 64% 55.2M 2s 2024-08-25T19:47:52.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108500K .......... .......... .......... .......... .......... 64% 33.6M 2s 2024-08-25T19:47:52.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108550K .......... .......... .......... .......... .......... 64% 26.2M 2s 2024-08-25T19:47:52.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108600K .......... .......... .......... .......... .......... 64% 22.5M 2s 2024-08-25T19:47:52.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108650K .......... .......... .......... .......... .......... 64% 53.4M 2s 2024-08-25T19:47:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108700K .......... .......... .......... .......... .......... 64% 23.0M 2s 2024-08-25T19:47:52.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108750K .......... .......... .......... .......... .......... 64% 23.6M 2s 2024-08-25T19:47:52.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108800K .......... .......... .......... .......... .......... 64% 40.0M 2s 2024-08-25T19:47:52.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108850K .......... .......... .......... .......... .......... 64% 22.9M 2s 2024-08-25T19:47:52.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108900K .......... .......... .......... .......... .......... 64% 24.2M 2s 2024-08-25T19:47:52.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:108950K .......... .......... .......... .......... .......... 64% 45.1M 2s 2024-08-25T19:47:52.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109000K .......... .......... .......... .......... .......... 64% 36.4M 2s 2024-08-25T19:47:52.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109050K .......... .......... .......... .......... .......... 64% 26.5M 2s 2024-08-25T19:47:52.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109100K .......... .......... .......... .......... .......... 64% 21.1M 2s 2024-08-25T19:47:52.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109150K .......... .......... .......... .......... .......... 64% 47.4M 2s 2024-08-25T19:47:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109200K .......... .......... .......... .......... .......... 64% 29.0M 2s 2024-08-25T19:47:52.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109250K .......... .......... .......... .......... .......... 64% 22.9M 2s 2024-08-25T19:47:52.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109300K .......... .......... .......... .......... .......... 64% 33.3M 2s 2024-08-25T19:47:52.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109350K .......... .......... .......... .......... .......... 64% 26.3M 2s 2024-08-25T19:47:52.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109400K .......... .......... .......... .......... .......... 64% 46.6M 2s 2024-08-25T19:47:52.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109450K .......... .......... .......... .......... .......... 64% 31.3M 2s 2024-08-25T19:47:52.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109500K .......... .......... .......... .......... .......... 64% 19.5M 2s 2024-08-25T19:47:52.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109550K .......... .......... .......... .......... .......... 65% 23.5M 2s 2024-08-25T19:47:52.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109600K .......... .......... .......... .......... .......... 65% 39.3M 2s 2024-08-25T19:47:52.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109650K .......... .......... .......... .......... .......... 65% 24.4M 2s 2024-08-25T19:47:52.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109700K .......... .......... .......... .......... .......... 65% 46.6M 2s 2024-08-25T19:47:52.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109750K .......... .......... .......... .......... .......... 65% 20.4M 2s 2024-08-25T19:47:52.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109800K .......... .......... .......... .......... .......... 65% 57.4M 2s 2024-08-25T19:47:52.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109850K .......... .......... .......... .......... .......... 65% 24.1M 2s 2024-08-25T19:47:52.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109900K .......... .......... .......... .......... .......... 65% 43.4M 2s 2024-08-25T19:47:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:109950K .......... .......... .......... .......... .......... 65% 23.1M 2s 2024-08-25T19:47:52.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110000K .......... .......... .......... .......... .......... 65% 25.3M 2s 2024-08-25T19:47:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110050K .......... .......... .......... .......... .......... 65% 25.4M 2s 2024-08-25T19:47:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110100K .......... .......... .......... .......... .......... 65% 33.0M 2s 2024-08-25T19:47:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110150K .......... .......... .......... .......... .......... 65% 21.9M 2s 2024-08-25T19:47:52.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110200K .......... .......... .......... .......... .......... 65% 43.9M 2s 2024-08-25T19:47:52.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110250K .......... .......... .......... .......... .......... 65% 30.5M 2s 2024-08-25T19:47:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110300K .......... .......... .......... .......... .......... 65% 36.0M 2s 2024-08-25T19:47:52.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110350K .......... .......... .......... .......... .......... 65% 22.0M 2s 2024-08-25T19:47:52.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110400K .......... .......... .......... .......... .......... 65% 25.5M 2s 2024-08-25T19:47:52.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110450K .......... .......... .......... .......... .......... 65% 45.0M 2s 2024-08-25T19:47:52.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110500K .......... .......... .......... .......... .......... 65% 23.0M 2s 2024-08-25T19:47:52.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110550K .......... .......... .......... .......... .......... 65% 24.0M 2s 2024-08-25T19:47:52.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110600K .......... .......... .......... .......... .......... 65% 40.6M 2s 2024-08-25T19:47:52.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110650K .......... .......... .......... .......... .......... 65% 26.4M 2s 2024-08-25T19:47:52.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110700K .......... .......... .......... .......... .......... 65% 47.3M 2s 2024-08-25T19:47:52.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110750K .......... .......... .......... .......... .......... 65% 17.3M 2s 2024-08-25T19:47:52.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110800K .......... .......... .......... .......... .......... 65% 25.1M 2s 2024-08-25T19:47:52.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110850K .......... .......... .......... .......... .......... 65% 40.9M 2s 2024-08-25T19:47:52.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110900K .......... .......... .......... .......... .......... 65% 25.9M 2s 2024-08-25T19:47:52.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:110950K .......... .......... .......... .......... .......... 65% 35.5M 2s 2024-08-25T19:47:52.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111000K .......... .......... .......... .......... .......... 65% 28.0M 2s 2024-08-25T19:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111050K .......... .......... .......... .......... .......... 65% 20.6M 2s 2024-08-25T19:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111100K .......... .......... .......... .......... .......... 65% 47.2M 2s 2024-08-25T19:47:52.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111150K .......... .......... .......... .......... .......... 65% 24.3M 2s 2024-08-25T19:47:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111200K .......... .......... .......... .......... .......... 65% 20.6M 2s 2024-08-25T19:47:52.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111250K .......... .......... .......... .......... .......... 66% 35.7M 2s 2024-08-25T19:47:52.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111300K .......... .......... .......... .......... .......... 66% 38.6M 2s 2024-08-25T19:47:52.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111350K .......... .......... .......... .......... .......... 66% 28.8M 2s 2024-08-25T19:47:52.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111400K .......... .......... .......... .......... .......... 66% 22.4M 2s 2024-08-25T19:47:52.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111450K .......... .......... .......... .......... .......... 66% 54.8M 2s 2024-08-25T19:47:52.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111500K .......... .......... .......... .......... .......... 66% 20.1M 2s 2024-08-25T19:47:52.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111550K .......... .......... .......... .......... .......... 66% 26.0M 2s 2024-08-25T19:47:52.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111600K .......... .......... .......... .......... .......... 66% 31.5M 2s 2024-08-25T19:47:52.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111650K .......... .......... .......... .......... .......... 66% 28.9M 2s 2024-08-25T19:47:52.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111700K .......... .......... .......... .......... .......... 66% 49.2M 2s 2024-08-25T19:47:52.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111750K .......... .......... .......... .......... .......... 66% 23.3M 2s 2024-08-25T19:47:52.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111800K .......... .......... .......... .......... .......... 66% 37.2M 2s 2024-08-25T19:47:52.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111850K .......... .......... .......... .......... .......... 66% 23.3M 2s 2024-08-25T19:47:52.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111900K .......... .......... .......... .......... .......... 66% 23.7M 2s 2024-08-25T19:47:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:111950K .......... .......... .......... .......... .......... 66% 26.0M 2s 2024-08-25T19:47:52.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112000K .......... .......... .......... .......... .......... 66% 47.3M 2s 2024-08-25T19:47:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112050K .......... .......... .......... .......... .......... 66% 21.3M 2s 2024-08-25T19:47:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112100K .......... .......... .......... .......... .......... 66% 33.3M 2s 2024-08-25T19:47:52.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112150K .......... .......... .......... .......... .......... 66% 24.1M 2s 2024-08-25T19:47:52.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112200K .......... .......... .......... .......... .......... 66% 38.8M 2s 2024-08-25T19:47:52.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112250K .......... .......... .......... .......... .......... 66% 27.8M 2s 2024-08-25T19:47:52.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112300K .......... .......... .......... .......... .......... 66% 38.8M 2s 2024-08-25T19:47:52.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112350K .......... .......... .......... .......... .......... 66% 22.9M 2s 2024-08-25T19:47:52.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112400K .......... .......... .......... .......... .......... 66% 23.1M 2s 2024-08-25T19:47:52.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112450K .......... .......... .......... .......... .......... 66% 24.4M 2s 2024-08-25T19:47:52.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112500K .......... .......... .......... .......... .......... 66% 41.3M 2s 2024-08-25T19:47:52.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112550K .......... .......... .......... .......... .......... 66% 24.0M 2s 2024-08-25T19:47:52.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112600K .......... .......... .......... .......... .......... 66% 43.1M 2s 2024-08-25T19:47:52.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112650K .......... .......... .......... .......... .......... 66% 22.5M 2s 2024-08-25T19:47:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112700K .......... .......... .......... .......... .......... 66% 38.7M 2s 2024-08-25T19:47:52.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112750K .......... .......... .......... .......... .......... 66% 23.0M 2s 2024-08-25T19:47:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112800K .......... .......... .......... .......... .......... 66% 27.0M 2s 2024-08-25T19:47:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112850K .......... .......... .......... .......... .......... 66% 37.9M 2s 2024-08-25T19:47:52.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112900K .......... .......... .......... .......... .......... 67% 21.4M 2s 2024-08-25T19:47:52.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:112950K .......... .......... .......... .......... .......... 67% 23.7M 2s 2024-08-25T19:47:52.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113000K .......... .......... .......... .......... .......... 67% 47.4M 2s 2024-08-25T19:47:52.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113050K .......... .......... .......... .......... .......... 67% 23.1M 2s 2024-08-25T19:47:52.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113100K .......... .......... .......... .......... .......... 67% 41.0M 2s 2024-08-25T19:47:52.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113150K .......... .......... .......... .......... .......... 67% 21.3M 2s 2024-08-25T19:47:52.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113200K .......... .......... .......... .......... .......... 67% 23.7M 2s 2024-08-25T19:47:52.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113250K .......... .......... .......... .......... .......... 67% 52.5M 2s 2024-08-25T19:47:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113300K .......... .......... .......... .......... .......... 67% 25.0M 2s 2024-08-25T19:47:52.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113350K .......... .......... .......... .......... .......... 67% 38.3M 2s 2024-08-25T19:47:52.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113400K .......... .......... .......... .......... .......... 67% 22.8M 2s 2024-08-25T19:47:52.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113450K .......... .......... .......... .......... .......... 67% 24.4M 2s 2024-08-25T19:47:52.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113500K .......... .......... .......... .......... .......... 67% 52.1M 2s 2024-08-25T19:47:52.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113550K .......... .......... .......... .......... .......... 67% 23.2M 2s 2024-08-25T19:47:52.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113600K .......... .......... .......... .......... .......... 67% 23.8M 2s 2024-08-25T19:47:52.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113650K .......... .......... .......... .......... .......... 67% 38.6M 2s 2024-08-25T19:47:52.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113700K .......... .......... .......... .......... .......... 67% 24.3M 2s 2024-08-25T19:47:52.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113750K .......... .......... .......... .......... .......... 67% 48.5M 2s 2024-08-25T19:47:52.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113800K .......... .......... .......... .......... .......... 67% 24.0M 2s 2024-08-25T19:47:52.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113850K .......... .......... .......... .......... .......... 67% 31.2M 2s 2024-08-25T19:47:52.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113900K .......... .......... .......... .......... .......... 67% 16.2M 2s 2024-08-25T19:47:52.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:113950K .......... .......... .......... .......... .......... 67% 26.2M 2s 2024-08-25T19:47:52.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114000K .......... .......... .......... .......... .......... 67% 10.8M 2s 2024-08-25T19:47:52.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114050K .......... .......... .......... .......... .......... 67% 60.3M 2s 2024-08-25T19:47:52.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114100K .......... .......... .......... .......... .......... 67% 22.5M 2s 2024-08-25T19:47:52.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114150K .......... .......... .......... .......... .......... 67% 40.2M 2s 2024-08-25T19:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114200K .......... .......... .......... .......... .......... 67% 17.1M 2s 2024-08-25T19:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114250K .......... .......... .......... .......... .......... 67% 65.5M 2s 2024-08-25T19:47:52.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114300K .......... .......... .......... .......... .......... 67% 20.7M 2s 2024-08-25T19:47:52.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114350K .......... .......... .......... .......... .......... 67% 23.7M 2s 2024-08-25T19:47:52.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114400K .......... .......... .......... .......... .......... 67% 38.6M 2s 2024-08-25T19:47:52.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114450K .......... .......... .......... .......... .......... 67% 13.2M 2s 2024-08-25T19:47:52.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114500K .......... .......... .......... .......... .......... 67% 42.4M 2s 2024-08-25T19:47:52.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114550K .......... .......... .......... .......... .......... 67% 25.8M 2s 2024-08-25T19:47:52.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114600K .......... .......... .......... .......... .......... 68% 34.4M 2s 2024-08-25T19:47:52.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114650K .......... .......... .......... .......... .......... 68% 22.2M 2s 2024-08-25T19:47:52.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114700K .......... .......... .......... .......... .......... 68% 24.0M 2s 2024-08-25T19:47:52.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114750K .......... .......... .......... .......... .......... 68% 26.6M 2s 2024-08-25T19:47:52.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114800K .......... .......... .......... .......... .......... 68% 36.9M 2s 2024-08-25T19:47:52.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114850K .......... .......... .......... .......... .......... 68% 28.0M 2s 2024-08-25T19:47:52.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114900K .......... .......... .......... .......... .......... 68% 30.2M 2s 2024-08-25T19:47:52.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:114950K .......... .......... .......... .......... .......... 68% 29.5M 2s 2024-08-25T19:47:52.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115000K .......... .......... .......... .......... .......... 68% 47.0M 2s 2024-08-25T19:47:52.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115050K .......... .......... .......... .......... .......... 68% 22.5M 2s 2024-08-25T19:47:52.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115100K .......... .......... .......... .......... .......... 68% 40.2M 2s 2024-08-25T19:47:52.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115150K .......... .......... .......... .......... .......... 68% 21.2M 2s 2024-08-25T19:47:52.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115200K .......... .......... .......... .......... .......... 68% 24.9M 2s 2024-08-25T19:47:52.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115250K .......... .......... .......... .......... .......... 68% 25.1M 2s 2024-08-25T19:47:52.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115300K .......... .......... .......... .......... .......... 68% 40.0M 2s 2024-08-25T19:47:52.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115350K .......... .......... .......... .......... .......... 68% 27.9M 2s 2024-08-25T19:47:52.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115400K .......... .......... .......... .......... .......... 68% 32.3M 2s 2024-08-25T19:47:52.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115450K .......... .......... .......... .......... .......... 68% 29.9M 2s 2024-08-25T19:47:52.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115500K .......... .......... .......... .......... .......... 68% 44.7M 2s 2024-08-25T19:47:52.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115550K .......... .......... .......... .......... .......... 68% 23.9M 2s 2024-08-25T19:47:52.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115600K .......... .......... .......... .......... .......... 68% 26.2M 2s 2024-08-25T19:47:52.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115650K .......... .......... .......... .......... .......... 68% 30.3M 2s 2024-08-25T19:47:52.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115700K .......... .......... .......... .......... .......... 68% 24.9M 2s 2024-08-25T19:47:52.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115750K .......... .......... .......... .......... .......... 68% 27.6M 2s 2024-08-25T19:47:52.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115800K .......... .......... .......... .......... .......... 68% 39.9M 2s 2024-08-25T19:47:52.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115850K .......... .......... .......... .......... .......... 68% 23.2M 2s 2024-08-25T19:47:52.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115900K .......... .......... .......... .......... .......... 68% 46.4M 2s 2024-08-25T19:47:52.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:115950K .......... .......... .......... .......... .......... 68% 24.0M 2s 2024-08-25T19:47:52.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116000K .......... .......... .......... .......... .......... 68% 22.8M 2s 2024-08-25T19:47:52.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116050K .......... .......... .......... .......... .......... 68% 41.0M 2s 2024-08-25T19:47:52.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116100K .......... .......... .......... .......... .......... 68% 26.5M 2s 2024-08-25T19:47:52.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116150K .......... .......... .......... .......... .......... 68% 36.5M 2s 2024-08-25T19:47:52.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116200K .......... .......... .......... .......... .......... 68% 27.0M 2s 2024-08-25T19:47:52.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116250K .......... .......... .......... .......... .......... 68% 43.6M 2s 2024-08-25T19:47:52.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116300K .......... .......... .......... .......... .......... 69% 21.5M 2s 2024-08-25T19:47:52.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116350K .......... .......... .......... .......... .......... 69% 24.8M 2s 2024-08-25T19:47:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116400K .......... .......... .......... .......... .......... 69% 30.2M 2s 2024-08-25T19:47:52.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116450K .......... .......... .......... .......... .......... 69% 28.5M 2s 2024-08-25T19:47:52.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116500K .......... .......... .......... .......... .......... 69% 29.0M 2s 2024-08-25T19:47:52.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116550K .......... .......... .......... .......... .......... 69% 34.2M 2s 2024-08-25T19:47:52.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116600K .......... .......... .......... .......... .......... 69% 26.0M 2s 2024-08-25T19:47:52.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116650K .......... .......... .......... .......... .......... 69% 22.2M 2s 2024-08-25T19:47:52.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116700K .......... .......... .......... .......... .......... 69% 37.2M 2s 2024-08-25T19:47:52.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116750K .......... .......... .......... .......... .......... 69% 5.09M 2s 2024-08-25T19:47:52.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116800K .......... .......... .......... .......... .......... 69% 30.5M 2s 2024-08-25T19:47:52.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116850K .......... .......... .......... .......... .......... 69% 23.6M 2s 2024-08-25T19:47:52.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116900K .......... .......... .......... .......... .......... 69% 36.2M 2s 2024-08-25T19:47:52.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:116950K .......... .......... .......... .......... .......... 69% 19.6M 2s 2024-08-25T19:47:52.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117000K .......... .......... .......... .......... .......... 69% 33.8M 2s 2024-08-25T19:47:52.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117050K .......... .......... .......... .......... .......... 69% 37.3M 2s 2024-08-25T19:47:52.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117100K .......... .......... .......... .......... .......... 69% 12.4M 2s 2024-08-25T19:47:52.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117150K .......... .......... .......... .......... .......... 69% 26.1M 2s 2024-08-25T19:47:52.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117200K .......... .......... .......... .......... .......... 69% 41.9M 2s 2024-08-25T19:47:52.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117250K .......... .......... .......... .......... .......... 69% 24.3M 2s 2024-08-25T19:47:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117300K .......... .......... .......... .......... .......... 69% 44.1M 2s 2024-08-25T19:47:52.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117350K .......... .......... .......... .......... .......... 69% 22.1M 2s 2024-08-25T19:47:52.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117400K .......... .......... .......... .......... .......... 69% 41.2M 2s 2024-08-25T19:47:52.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117450K .......... .......... .......... .......... .......... 69% 19.7M 2s 2024-08-25T19:47:52.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117500K .......... .......... .......... .......... .......... 69% 23.2M 2s 2024-08-25T19:47:52.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117550K .......... .......... .......... .......... .......... 69% 37.0M 2s 2024-08-25T19:47:52.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117600K .......... .......... .......... .......... .......... 69% 28.3M 2s 2024-08-25T19:47:52.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117650K .......... .......... .......... .......... .......... 69% 23.0M 2s 2024-08-25T19:47:52.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117700K .......... .......... .......... .......... .......... 69% 40.9M 2s 2024-08-25T19:47:52.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117750K .......... .......... .......... .......... .......... 69% 35.6M 2s 2024-08-25T19:47:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117800K .......... .......... .......... .......... .......... 69% 28.0M 2s 2024-08-25T19:47:52.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117850K .......... .......... .......... .......... .......... 69% 24.1M 2s 2024-08-25T19:47:52.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117900K .......... .......... .......... .......... .......... 69% 39.5M 2s 2024-08-25T19:47:52.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:117950K .......... .......... .......... .......... .......... 70% 23.8M 2s 2024-08-25T19:47:52.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118000K .......... .......... .......... .......... .......... 70% 21.8M 2s 2024-08-25T19:47:52.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118050K .......... .......... .......... .......... .......... 70% 26.2M 2s 2024-08-25T19:47:52.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118100K .......... .......... .......... .......... .......... 70% 37.9M 2s 2024-08-25T19:47:52.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118150K .......... .......... .......... .......... .......... 70% 27.2M 2s 2024-08-25T19:47:52.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118200K .......... .......... .......... .......... .......... 70% 33.2M 2s 2024-08-25T19:47:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118250K .......... .......... .......... .......... .......... 70% 22.2M 2s 2024-08-25T19:47:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118300K .......... .......... .......... .......... .......... 70% 44.6M 2s 2024-08-25T19:47:52.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118350K .......... .......... .......... .......... .......... 70% 23.8M 2s 2024-08-25T19:47:52.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118400K .......... .......... .......... .......... .......... 70% 25.8M 2s 2024-08-25T19:47:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118450K .......... .......... .......... .......... .......... 70% 22.9M 2s 2024-08-25T19:47:52.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118500K .......... .......... .......... .......... .......... 70% 36.4M 2s 2024-08-25T19:47:52.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118550K .......... .......... .......... .......... .......... 70% 23.1M 2s 2024-08-25T19:47:52.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118600K .......... .......... .......... .......... .......... 70% 58.0M 2s 2024-08-25T19:47:52.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118650K .......... .......... .......... .......... .......... 70% 22.2M 2s 2024-08-25T19:47:52.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118700K .......... .......... .......... .......... .......... 70% 41.3M 2s 2024-08-25T19:47:52.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118750K .......... .......... .......... .......... .......... 70% 23.0M 2s 2024-08-25T19:47:52.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118800K .......... .......... .......... .......... .......... 70% 34.1M 2s 2024-08-25T19:47:52.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118850K .......... .......... .......... .......... .......... 70% 28.5M 2s 2024-08-25T19:47:52.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118900K .......... .......... .......... .......... .......... 70% 23.1M 2s 2024-08-25T19:47:52.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:118950K .......... .......... .......... .......... .......... 70% 42.8M 2s 2024-08-25T19:47:52.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119000K .......... .......... .......... .......... .......... 70% 30.1M 2s 2024-08-25T19:47:52.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119050K .......... .......... .......... .......... .......... 70% 32.7M 2s 2024-08-25T19:47:52.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119100K .......... .......... .......... .......... .......... 70% 23.3M 2s 2024-08-25T19:47:52.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119150K .......... .......... .......... .......... .......... 70% 23.0M 2s 2024-08-25T19:47:52.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119200K .......... .......... .......... .......... .......... 70% 25.3M 2s 2024-08-25T19:47:52.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119250K .......... .......... .......... .......... .......... 70% 39.2M 2s 2024-08-25T19:47:52.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119300K .......... .......... .......... .......... .......... 70% 24.0M 2s 2024-08-25T19:47:52.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119350K .......... .......... .......... .......... .......... 70% 41.5M 2s 2024-08-25T19:47:52.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119400K .......... .......... .......... .......... .......... 70% 22.6M 2s 2024-08-25T19:47:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119450K .......... .......... .......... .......... .......... 70% 26.8M 2s 2024-08-25T19:47:52.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119500K .......... .......... .......... .......... .......... 70% 40.2M 2s 2024-08-25T19:47:52.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119550K .......... .......... .......... .......... .......... 70% 18.3M 2s 2024-08-25T19:47:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119600K .......... .......... .......... .......... .......... 70% 44.2M 2s 2024-08-25T19:47:52.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119650K .......... .......... .......... .......... .......... 71% 22.3M 2s 2024-08-25T19:47:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119700K .......... .......... .......... .......... .......... 71% 26.7M 2s 2024-08-25T19:47:52.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119750K .......... .......... .......... .......... .......... 71% 36.0M 2s 2024-08-25T19:47:52.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119800K .......... .......... .......... .......... .......... 71% 22.4M 2s 2024-08-25T19:47:52.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119850K .......... .......... .......... .......... .......... 71% 42.8M 2s 2024-08-25T19:47:52.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119900K .......... .......... .......... .......... .......... 71% 34.2M 2s 2024-08-25T19:47:52.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:119950K .......... .......... .......... .......... .......... 71% 18.3M 2s 2024-08-25T19:47:52.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120000K .......... .......... .......... .......... .......... 71% 33.9M 2s 2024-08-25T19:47:52.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120050K .......... .......... .......... .......... .......... 71% 29.1M 2s 2024-08-25T19:47:52.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120100K .......... .......... .......... .......... .......... 71% 44.5M 2s 2024-08-25T19:47:52.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120150K .......... .......... .......... .......... .......... 71% 23.8M 2s 2024-08-25T19:47:52.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120200K .......... .......... .......... .......... .......... 71% 40.0M 2s 2024-08-25T19:47:52.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120250K .......... .......... .......... .......... .......... 71% 21.1M 2s 2024-08-25T19:47:52.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120300K .......... .......... .......... .......... .......... 71% 15.8M 2s 2024-08-25T19:47:52.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120350K .......... .......... .......... .......... .......... 71% 35.1M 2s 2024-08-25T19:47:52.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120400K .......... .......... .......... .......... .......... 71% 39.6M 2s 2024-08-25T19:47:52.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120450K .......... .......... .......... .......... .......... 71% 21.5M 2s 2024-08-25T19:47:52.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120500K .......... .......... .......... .......... .......... 71% 38.7M 2s 2024-08-25T19:47:52.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120550K .......... .......... .......... .......... .......... 71% 24.7M 2s 2024-08-25T19:47:52.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120600K .......... .......... .......... .......... .......... 71% 43.9M 2s 2024-08-25T19:47:52.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120650K .......... .......... .......... .......... .......... 71% 10.9M 2s 2024-08-25T19:47:52.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120700K .......... .......... .......... .......... .......... 71% 165M 2s 2024-08-25T19:47:52.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120750K .......... .......... .......... .......... .......... 71% 11.9M 2s 2024-08-25T19:47:52.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120800K .......... .......... .......... .......... .......... 71% 18.2M 2s 2024-08-25T19:47:52.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120850K .......... .......... .......... .......... .......... 71% 20.3M 2s 2024-08-25T19:47:52.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120900K .......... .......... .......... .......... .......... 71% 47.1M 2s 2024-08-25T19:47:52.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:120950K .......... .......... .......... .......... .......... 71% 22.2M 2s 2024-08-25T19:47:52.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121000K .......... .......... .......... .......... .......... 71% 42.0M 2s 2024-08-25T19:47:52.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121050K .......... .......... .......... .......... .......... 71% 30.1M 2s 2024-08-25T19:47:52.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121100K .......... .......... .......... .......... .......... 71% 38.2M 2s 2024-08-25T19:47:52.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121150K .......... .......... .......... .......... .......... 71% 21.6M 2s 2024-08-25T19:47:52.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121200K .......... .......... .......... .......... .......... 71% 20.9M 2s 2024-08-25T19:47:52.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121250K .......... .......... .......... .......... .......... 71% 43.2M 2s 2024-08-25T19:47:52.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121300K .......... .......... .......... .......... .......... 71% 23.0M 2s 2024-08-25T19:47:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121350K .......... .......... .......... .......... .......... 72% 23.0M 2s 2024-08-25T19:47:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121400K .......... .......... .......... .......... .......... 72% 51.0M 2s 2024-08-25T19:47:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121450K .......... .......... .......... .......... .......... 72% 38.1M 2s 2024-08-25T19:47:52.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121500K .......... .......... .......... .......... .......... 72% 25.4M 2s 2024-08-25T19:47:52.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121550K .......... .......... .......... .......... .......... 72% 23.5M 2s 2024-08-25T19:47:52.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121600K .......... .......... .......... .......... .......... 72% 26.3M 2s 2024-08-25T19:47:52.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121650K .......... .......... .......... .......... .......... 72% 30.7M 2s 2024-08-25T19:47:52.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121700K .......... .......... .......... .......... .......... 72% 30.1M 2s 2024-08-25T19:47:52.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121750K .......... .......... .......... .......... .......... 72% 40.9M 2s 2024-08-25T19:47:52.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121800K .......... .......... .......... .......... .......... 72% 22.1M 2s 2024-08-25T19:47:52.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121850K .......... .......... .......... .......... .......... 72% 51.1M 2s 2024-08-25T19:47:52.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121900K .......... .......... .......... .......... .......... 72% 24.6M 2s 2024-08-25T19:47:52.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:121950K .......... .......... .......... .......... .......... 72% 21.9M 2s 2024-08-25T19:47:52.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122000K .......... .......... .......... .......... .......... 72% 22.5M 2s 2024-08-25T19:47:52.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122050K .......... .......... .......... .......... .......... 72% 56.0M 2s 2024-08-25T19:47:52.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122100K .......... .......... .......... .......... .......... 72% 21.0M 2s 2024-08-25T19:47:52.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122150K .......... .......... .......... .......... .......... 72% 42.5M 2s 2024-08-25T19:47:52.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122200K .......... .......... .......... .......... .......... 72% 28.9M 2s 2024-08-25T19:47:52.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122250K .......... .......... .......... .......... .......... 72% 35.2M 2s 2024-08-25T19:47:52.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122300K .......... .......... .......... .......... .......... 72% 24.8M 2s 2024-08-25T19:47:52.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122350K .......... .......... .......... .......... .......... 72% 23.1M 2s 2024-08-25T19:47:52.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122400K .......... .......... .......... .......... .......... 72% 46.2M 2s 2024-08-25T19:47:52.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122450K .......... .......... .......... .......... .......... 72% 24.4M 2s 2024-08-25T19:47:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122500K .......... .......... .......... .......... .......... 72% 23.3M 2s 2024-08-25T19:47:52.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122550K .......... .......... .......... .......... .......... 72% 41.4M 2s 2024-08-25T19:47:52.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122600K .......... .......... .......... .......... .......... 72% 24.2M 2s 2024-08-25T19:47:52.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122650K .......... .......... .......... .......... .......... 72% 43.7M 2s 2024-08-25T19:47:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122700K .......... .......... .......... .......... .......... 72% 26.0M 2s 2024-08-25T19:47:52.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122750K .......... .......... .......... .......... .......... 72% 22.5M 2s 2024-08-25T19:47:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122800K .......... .......... .......... .......... .......... 72% 46.1M 2s 2024-08-25T19:47:52.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122850K .......... .......... .......... .......... .......... 72% 22.7M 2s 2024-08-25T19:47:52.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122900K .......... .......... .......... .......... .......... 72% 37.1M 2s 2024-08-25T19:47:53.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:122950K .......... .......... .......... .......... .......... 72% 22.6M 2s 2024-08-25T19:47:53.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123000K .......... .......... .......... .......... .......... 72% 55.3M 2s 2024-08-25T19:47:53.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123050K .......... .......... .......... .......... .......... 73% 25.8M 2s 2024-08-25T19:47:53.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123100K .......... .......... .......... .......... .......... 73% 21.2M 2s 2024-08-25T19:47:53.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123150K .......... .......... .......... .......... .......... 73% 25.8M 2s 2024-08-25T19:47:53.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123200K .......... .......... .......... .......... .......... 73% 46.2M 2s 2024-08-25T19:47:53.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123250K .......... .......... .......... .......... .......... 73% 24.3M 2s 2024-08-25T19:47:53.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123300K .......... .......... .......... .......... .......... 73% 39.8M 2s 2024-08-25T19:47:53.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123350K .......... .......... .......... .......... .......... 73% 21.6M 2s 2024-08-25T19:47:53.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123400K .......... .......... .......... .......... .......... 73% 50.1M 2s 2024-08-25T19:47:53.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123450K .......... .......... .......... .......... .......... 73% 24.7M 2s 2024-08-25T19:47:53.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123500K .......... .......... .......... .......... .......... 73% 37.8M 2s 2024-08-25T19:47:53.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123550K .......... .......... .......... .......... .......... 73% 22.7M 2s 2024-08-25T19:47:53.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123600K .......... .......... .......... .......... .......... 73% 26.5M 2s 2024-08-25T19:47:53.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123650K .......... .......... .......... .......... .......... 73% 23.2M 2s 2024-08-25T19:47:53.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123700K .......... .......... .......... .......... .......... 73% 43.1M 2s 2024-08-25T19:47:53.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123750K .......... .......... .......... .......... .......... 73% 22.5M 2s 2024-08-25T19:47:53.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123800K .......... .......... .......... .......... .......... 73% 38.9M 2s 2024-08-25T19:47:53.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123850K .......... .......... .......... .......... .......... 73% 27.0M 2s 2024-08-25T19:47:53.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123900K .......... .......... .......... .......... .......... 73% 13.9M 2s 2024-08-25T19:47:53.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:123950K .......... .......... .......... .......... .......... 73% 6.73M 2s 2024-08-25T19:47:53.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124000K .......... .......... .......... .......... .......... 73% 9.81M 2s 2024-08-25T19:47:53.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124050K .......... .......... .......... .......... .......... 73% 43.2M 2s 2024-08-25T19:47:53.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124100K .......... .......... .......... .......... .......... 73% 23.7M 2s 2024-08-25T19:47:53.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124150K .......... .......... .......... .......... .......... 73% 47.8M 2s 2024-08-25T19:47:53.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124200K .......... .......... .......... .......... .......... 73% 17.3M 2s 2024-08-25T19:47:53.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124250K .......... .......... .......... .......... .......... 73% 32.0M 2s 2024-08-25T19:47:53.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124300K .......... .......... .......... .......... .......... 73% 23.4M 2s 2024-08-25T19:47:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124350K .......... .......... .......... .......... .......... 73% 27.0M 2s 2024-08-25T19:47:53.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124400K .......... .......... .......... .......... .......... 73% 17.8M 2s 2024-08-25T19:47:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124450K .......... .......... .......... .......... .......... 73% 39.2M 2s 2024-08-25T19:47:53.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124500K .......... .......... .......... .......... .......... 73% 26.3M 2s 2024-08-25T19:47:53.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124550K .......... .......... .......... .......... .......... 73% 40.2M 2s 2024-08-25T19:47:53.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124600K .......... .......... .......... .......... .......... 73% 24.7M 2s 2024-08-25T19:47:53.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124650K .......... .......... .......... .......... .......... 73% 38.1M 2s 2024-08-25T19:47:53.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124700K .......... .......... .......... .......... .......... 74% 28.0M 2s 2024-08-25T19:47:53.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124750K .......... .......... .......... .......... .......... 74% 23.5M 2s 2024-08-25T19:47:53.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124800K .......... .......... .......... .......... .......... 74% 35.9M 2s 2024-08-25T19:47:53.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124850K .......... .......... .......... .......... .......... 74% 22.7M 2s 2024-08-25T19:47:53.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124900K .......... .......... .......... .......... .......... 74% 27.9M 2s 2024-08-25T19:47:53.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:124950K .......... .......... .......... .......... .......... 74% 45.4M 2s 2024-08-25T19:47:53.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125000K .......... .......... .......... .......... .......... 74% 21.4M 2s 2024-08-25T19:47:53.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125050K .......... .......... .......... .......... .......... 74% 54.2M 2s 2024-08-25T19:47:53.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125100K .......... .......... .......... .......... .......... 74% 20.2M 2s 2024-08-25T19:47:53.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125150K .......... .......... .......... .......... .......... 74% 28.4M 2s 2024-08-25T19:47:53.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125200K .......... .......... .......... .......... .......... 74% 37.0M 2s 2024-08-25T19:47:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125250K .......... .......... .......... .......... .......... 74% 25.5M 2s 2024-08-25T19:47:53.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125300K .......... .......... .......... .......... .......... 74% 27.5M 2s 2024-08-25T19:47:53.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125350K .......... .......... .......... .......... .......... 74% 23.1M 2s 2024-08-25T19:47:53.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125400K .......... .......... .......... .......... .......... 74% 20.2M 2s 2024-08-25T19:47:53.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125450K .......... .......... .......... .......... .......... 74% 31.7M 2s 2024-08-25T19:47:53.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125500K .......... .......... .......... .......... .......... 74% 24.1M 2s 2024-08-25T19:47:53.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125550K .......... .......... .......... .......... .......... 74% 25.8M 2s 2024-08-25T19:47:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125600K .......... .......... .......... .......... .......... 74% 37.7M 2s 2024-08-25T19:47:53.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125650K .......... .......... .......... .......... .......... 74% 24.3M 2s 2024-08-25T19:47:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125700K .......... .......... .......... .......... .......... 74% 46.1M 2s 2024-08-25T19:47:53.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125750K .......... .......... .......... .......... .......... 74% 22.8M 2s 2024-08-25T19:47:53.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125800K .......... .......... .......... .......... .......... 74% 18.9M 2s 2024-08-25T19:47:53.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125850K .......... .......... .......... .......... .......... 74% 40.5M 2s 2024-08-25T19:47:53.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125900K .......... .......... .......... .......... .......... 74% 25.9M 2s 2024-08-25T19:47:53.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:125950K .......... .......... .......... .......... .......... 74% 23.1M 2s 2024-08-25T19:47:53.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126000K .......... .......... .......... .......... .......... 74% 49.2M 2s 2024-08-25T19:47:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126050K .......... .......... .......... .......... .......... 74% 22.1M 2s 2024-08-25T19:47:53.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126100K .......... .......... .......... .......... .......... 74% 32.9M 2s 2024-08-25T19:47:53.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126150K .......... .......... .......... .......... .......... 74% 23.3M 2s 2024-08-25T19:47:53.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126200K .......... .......... .......... .......... .......... 74% 45.3M 2s 2024-08-25T19:47:53.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126250K .......... .......... .......... .......... .......... 74% 23.9M 2s 2024-08-25T19:47:53.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126300K .......... .......... .......... .......... .......... 74% 53.5M 2s 2024-08-25T19:47:53.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126350K .......... .......... .......... .......... .......... 74% 16.2M 2s 2024-08-25T19:47:53.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126400K .......... .......... .......... .......... .......... 75% 40.4M 2s 2024-08-25T19:47:53.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126450K .......... .......... .......... .......... .......... 75% 22.9M 2s 2024-08-25T19:47:53.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126500K .......... .......... .......... .......... .......... 75% 56.4M 2s 2024-08-25T19:47:53.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126550K .......... .......... .......... .......... .......... 75% 16.1M 2s 2024-08-25T19:47:53.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126600K .......... .......... .......... .......... .......... 75% 37.4M 2s 2024-08-25T19:47:53.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126650K .......... .......... .......... .......... .......... 75% 21.6M 2s 2024-08-25T19:47:53.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126700K .......... .......... .......... .......... .......... 75% 47.0M 2s 2024-08-25T19:47:53.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126750K .......... .......... .......... .......... .......... 75% 22.5M 2s 2024-08-25T19:47:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126800K .......... .......... .......... .......... .......... 75% 24.4M 2s 2024-08-25T19:47:53.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126850K .......... .......... .......... .......... .......... 75% 37.9M 2s 2024-08-25T19:47:53.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126900K .......... .......... .......... .......... .......... 75% 24.1M 2s 2024-08-25T19:47:53.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:126950K .......... .......... .......... .......... .......... 75% 33.4M 2s 2024-08-25T19:47:53.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127000K .......... .......... .......... .......... .......... 75% 23.6M 2s 2024-08-25T19:47:53.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127050K .......... .......... .......... .......... .......... 75% 24.7M 2s 2024-08-25T19:47:53.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127100K .......... .......... .......... .......... .......... 75% 49.6M 2s 2024-08-25T19:47:53.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127150K .......... .......... .......... .......... .......... 75% 24.0M 2s 2024-08-25T19:47:53.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127200K .......... .......... .......... .......... .......... 75% 24.0M 2s 2024-08-25T19:47:53.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127250K .......... .......... .......... .......... .......... 75% 42.3M 2s 2024-08-25T19:47:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127300K .......... .......... .......... .......... .......... 75% 26.7M 2s 2024-08-25T19:47:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127350K .......... .......... .......... .......... .......... 75% 36.1M 2s 2024-08-25T19:47:53.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127400K .......... .......... .......... .......... .......... 75% 24.6M 2s 2024-08-25T19:47:53.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127450K .......... .......... .......... .......... .......... 75% 44.6M 2s 2024-08-25T19:47:53.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127500K .......... .......... .......... .......... .......... 75% 22.5M 2s 2024-08-25T19:47:53.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127550K .......... .......... .......... .......... .......... 75% 24.6M 2s 2024-08-25T19:47:53.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127600K .......... .......... .......... .......... .......... 75% 37.0M 2s 2024-08-25T19:47:53.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127650K .......... .......... .......... .......... .......... 75% 20.3M 2s 2024-08-25T19:47:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127700K .......... .......... .......... .......... .......... 75% 28.7M 2s 2024-08-25T19:47:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127750K .......... .......... .......... .......... .......... 75% 33.7M 2s 2024-08-25T19:47:53.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127800K .......... .......... .......... .......... .......... 75% 24.6M 2s 2024-08-25T19:47:53.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127850K .......... .......... .......... .......... .......... 75% 49.2M 2s 2024-08-25T19:47:53.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127900K .......... .......... .......... .......... .......... 75% 25.3M 2s 2024-08-25T19:47:53.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:127950K .......... .......... .......... .......... .......... 75% 24.0M 2s 2024-08-25T19:47:53.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128000K .......... .......... .......... .......... .......... 75% 31.3M 2s 2024-08-25T19:47:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128050K .......... .......... .......... .......... .......... 75% 24.7M 2s 2024-08-25T19:47:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128100K .......... .......... .......... .......... .......... 76% 37.6M 1s 2024-08-25T19:47:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128150K .......... .......... .......... .......... .......... 76% 23.8M 1s 2024-08-25T19:47:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128200K .......... .......... .......... .......... .......... 76% 21.7M 1s 2024-08-25T19:47:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128250K .......... .......... .......... .......... .......... 76% 39.0M 1s 2024-08-25T19:47:53.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128300K .......... .......... .......... .......... .......... 76% 28.9M 1s 2024-08-25T19:47:53.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128350K .......... .......... .......... .......... .......... 76% 20.2M 1s 2024-08-25T19:47:53.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128400K .......... .......... .......... .......... .......... 76% 41.0M 1s 2024-08-25T19:47:53.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128450K .......... .......... .......... .......... .......... 76% 13.1M 1s 2024-08-25T19:47:53.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128500K .......... .......... .......... .......... .......... 76% 46.1M 1s 2024-08-25T19:47:53.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128550K .......... .......... .......... .......... .......... 76% 21.8M 1s 2024-08-25T19:47:53.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128600K .......... .......... .......... .......... .......... 76% 38.7M 1s 2024-08-25T19:47:53.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128650K .......... .......... .......... .......... .......... 76% 27.8M 1s 2024-08-25T19:47:53.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128700K .......... .......... .......... .......... .......... 76% 38.2M 1s 2024-08-25T19:47:53.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128750K .......... .......... .......... .......... .......... 76% 20.9M 1s 2024-08-25T19:47:53.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128800K .......... .......... .......... .......... .......... 76% 39.8M 1s 2024-08-25T19:47:53.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128850K .......... .......... .......... .......... .......... 76% 19.5M 1s 2024-08-25T19:47:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128900K .......... .......... .......... .......... .......... 76% 36.4M 1s 2024-08-25T19:47:53.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:128950K .......... .......... .......... .......... .......... 76% 24.3M 1s 2024-08-25T19:47:53.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129000K .......... .......... .......... .......... .......... 76% 75.1M 1s 2024-08-25T19:47:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129050K .......... .......... .......... .......... .......... 76% 27.2M 1s 2024-08-25T19:47:53.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129100K .......... .......... .......... .......... .......... 76% 17.4M 1s 2024-08-25T19:47:53.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129150K .......... .......... .......... .......... .......... 76% 41.4M 1s 2024-08-25T19:47:53.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129200K .......... .......... .......... .......... .......... 76% 23.9M 1s 2024-08-25T19:47:53.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129250K .......... .......... .......... .......... .......... 76% 50.3M 1s 2024-08-25T19:47:53.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129300K .......... .......... .......... .......... .......... 76% 23.7M 1s 2024-08-25T19:47:53.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129350K .......... .......... .......... .......... .......... 76% 23.6M 1s 2024-08-25T19:47:53.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129400K .......... .......... .......... .......... .......... 76% 30.7M 1s 2024-08-25T19:47:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129450K .......... .......... .......... .......... .......... 76% 26.2M 1s 2024-08-25T19:47:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129500K .......... .......... .......... .......... .......... 76% 47.3M 1s 2024-08-25T19:47:53.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129550K .......... .......... .......... .......... .......... 76% 14.0M 1s 2024-08-25T19:47:53.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129600K .......... .......... .......... .......... .......... 76% 24.6M 1s 2024-08-25T19:47:53.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129650K .......... .......... .......... .......... .......... 76% 39.5M 1s 2024-08-25T19:47:53.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129700K .......... .......... .......... .......... .......... 76% 9.07M 1s 2024-08-25T19:47:53.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129750K .......... .......... .......... .......... .......... 77% 28.2M 1s 2024-08-25T19:47:53.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129800K .......... .......... .......... .......... .......... 77% 18.0M 1s 2024-08-25T19:47:53.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129850K .......... .......... .......... .......... .......... 77% 26.3M 1s 2024-08-25T19:47:53.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129900K .......... .......... .......... .......... .......... 77% 31.1M 1s 2024-08-25T19:47:53.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:129950K .......... .......... .......... .......... .......... 77% 20.2M 1s 2024-08-25T19:47:53.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130000K .......... .......... .......... .......... .......... 77% 22.8M 1s 2024-08-25T19:47:53.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130050K .......... .......... .......... .......... .......... 77% 41.3M 1s 2024-08-25T19:47:53.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130100K .......... .......... .......... .......... .......... 77% 10.9M 1s 2024-08-25T19:47:53.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130150K .......... .......... .......... .......... .......... 77% 36.4M 1s 2024-08-25T19:47:53.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130200K .......... .......... .......... .......... .......... 77% 20.5M 1s 2024-08-25T19:47:53.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130250K .......... .......... .......... .......... .......... 77% 44.6M 1s 2024-08-25T19:47:53.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130300K .......... .......... .......... .......... .......... 77% 19.4M 1s 2024-08-25T19:47:53.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130350K .......... .......... .......... .......... .......... 77% 11.4M 1s 2024-08-25T19:47:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130400K .......... .......... .......... .......... .......... 77% 38.0M 1s 2024-08-25T19:47:53.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130450K .......... .......... .......... .......... .......... 77% 22.9M 1s 2024-08-25T19:47:53.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130500K .......... .......... .......... .......... .......... 77% 24.5M 1s 2024-08-25T19:47:53.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130550K .......... .......... .......... .......... .......... 77% 48.5M 1s 2024-08-25T19:47:53.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130600K .......... .......... .......... .......... .......... 77% 27.1M 1s 2024-08-25T19:47:53.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130650K .......... .......... .......... .......... .......... 77% 35.3M 1s 2024-08-25T19:47:53.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130700K .......... .......... .......... .......... .......... 77% 24.2M 1s 2024-08-25T19:47:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130750K .......... .......... .......... .......... .......... 77% 24.1M 1s 2024-08-25T19:47:53.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130800K .......... .......... .......... .......... .......... 77% 31.9M 1s 2024-08-25T19:47:53.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130850K .......... .......... .......... .......... .......... 77% 30.3M 1s 2024-08-25T19:47:53.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130900K .......... .......... .......... .......... .......... 77% 39.1M 1s 2024-08-25T19:47:53.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:130950K .......... .......... .......... .......... .......... 77% 24.0M 1s 2024-08-25T19:47:53.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131000K .......... .......... .......... .......... .......... 77% 23.3M 1s 2024-08-25T19:47:53.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131050K .......... .......... .......... .......... .......... 77% 50.1M 1s 2024-08-25T19:47:53.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131100K .......... .......... .......... .......... .......... 77% 26.0M 1s 2024-08-25T19:47:53.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131150K .......... .......... .......... .......... .......... 77% 20.1M 1s 2024-08-25T19:47:53.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131200K .......... .......... .......... .......... .......... 77% 43.6M 1s 2024-08-25T19:47:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131250K .......... .......... .......... .......... .......... 77% 20.8M 1s 2024-08-25T19:47:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131300K .......... .......... .......... .......... .......... 77% 52.3M 1s 2024-08-25T19:47:53.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131350K .......... .......... .......... .......... .......... 77% 23.6M 1s 2024-08-25T19:47:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131400K .......... .......... .......... .......... .......... 77% 29.9M 1s 2024-08-25T19:47:53.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131450K .......... .......... .......... .......... .......... 78% 30.9M 1s 2024-08-25T19:47:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131500K .......... .......... .......... .......... .......... 78% 45.3M 1s 2024-08-25T19:47:53.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131550K .......... .......... .......... .......... .......... 78% 11.7M 1s 2024-08-25T19:47:53.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131600K .......... .......... .......... .......... .......... 78% 40.1M 1s 2024-08-25T19:47:53.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131650K .......... .......... .......... .......... .......... 78% 24.9M 1s 2024-08-25T19:47:53.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131700K .......... .......... .......... .......... .......... 78% 41.7M 1s 2024-08-25T19:47:53.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131750K .......... .......... .......... .......... .......... 78% 23.8M 1s 2024-08-25T19:47:53.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131800K .......... .......... .......... .......... .......... 78% 48.5M 1s 2024-08-25T19:47:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131850K .......... .......... .......... .......... .......... 78% 24.5M 1s 2024-08-25T19:47:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131900K .......... .......... .......... .......... .......... 78% 39.2M 1s 2024-08-25T19:47:53.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:131950K .......... .......... .......... .......... .......... 78% 24.0M 1s 2024-08-25T19:47:53.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132000K .......... .......... .......... .......... .......... 78% 21.6M 1s 2024-08-25T19:47:53.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132050K .......... .......... .......... .......... .......... 78% 34.2M 1s 2024-08-25T19:47:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132100K .......... .......... .......... .......... .......... 78% 28.9M 1s 2024-08-25T19:47:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132150K .......... .......... .......... .......... .......... 78% 23.6M 1s 2024-08-25T19:47:53.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132200K .......... .......... .......... .......... .......... 78% 34.6M 1s 2024-08-25T19:47:53.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132250K .......... .......... .......... .......... .......... 78% 24.3M 1s 2024-08-25T19:47:53.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132300K .......... .......... .......... .......... .......... 78% 56.4M 1s 2024-08-25T19:47:53.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132350K .......... .......... .......... .......... .......... 78% 23.8M 1s 2024-08-25T19:47:53.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132400K .......... .......... .......... .......... .......... 78% 24.7M 1s 2024-08-25T19:47:53.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132450K .......... .......... .......... .......... .......... 78% 37.1M 1s 2024-08-25T19:47:53.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132500K .......... .......... .......... .......... .......... 78% 22.8M 1s 2024-08-25T19:47:53.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132550K .......... .......... .......... .......... .......... 78% 44.6M 1s 2024-08-25T19:47:53.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132600K .......... .......... .......... .......... .......... 78% 21.1M 1s 2024-08-25T19:47:53.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132650K .......... .......... .......... .......... .......... 78% 25.1M 1s 2024-08-25T19:47:53.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132700K .......... .......... .......... .......... .......... 78% 45.4M 1s 2024-08-25T19:47:53.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132750K .......... .......... .......... .......... .......... 78% 22.0M 1s 2024-08-25T19:47:53.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132800K .......... .......... .......... .......... .......... 78% 25.1M 1s 2024-08-25T19:47:53.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132850K .......... .......... .......... .......... .......... 78% 45.7M 1s 2024-08-25T19:47:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132900K .......... .......... .......... .......... .......... 78% 22.3M 1s 2024-08-25T19:47:53.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:132950K .......... .......... .......... .......... .......... 78% 33.8M 1s 2024-08-25T19:47:53.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133000K .......... .......... .......... .......... .......... 78% 15.9M 1s 2024-08-25T19:47:53.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133050K .......... .......... .......... .......... .......... 78% 42.2M 1s 2024-08-25T19:47:53.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133100K .......... .......... .......... .......... .......... 78% 23.6M 1s 2024-08-25T19:47:53.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133150K .......... .......... .......... .......... .......... 79% 22.9M 1s 2024-08-25T19:47:53.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133200K .......... .......... .......... .......... .......... 79% 43.2M 1s 2024-08-25T19:47:53.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133250K .......... .......... .......... .......... .......... 79% 22.5M 1s 2024-08-25T19:47:53.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133300K .......... .......... .......... .......... .......... 79% 21.9M 1s 2024-08-25T19:47:53.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133350K .......... .......... .......... .......... .......... 79% 44.2M 1s 2024-08-25T19:47:53.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133400K .......... .......... .......... .......... .......... 79% 26.0M 1s 2024-08-25T19:47:53.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133450K .......... .......... .......... .......... .......... 79% 41.2M 1s 2024-08-25T19:47:53.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133500K .......... .......... .......... .......... .......... 79% 21.8M 1s 2024-08-25T19:47:53.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133550K .......... .......... .......... .......... .......... 79% 23.6M 1s 2024-08-25T19:47:53.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133600K .......... .......... .......... .......... .......... 79% 48.3M 1s 2024-08-25T19:47:53.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133650K .......... .......... .......... .......... .......... 79% 19.9M 1s 2024-08-25T19:47:53.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133700K .......... .......... .......... .......... .......... 79% 38.4M 1s 2024-08-25T19:47:53.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133750K .......... .......... .......... .......... .......... 79% 23.9M 1s 2024-08-25T19:47:53.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133800K .......... .......... .......... .......... .......... 79% 45.6M 1s 2024-08-25T19:47:53.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133850K .......... .......... .......... .......... .......... 79% 25.0M 1s 2024-08-25T19:47:53.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133900K .......... .......... .......... .......... .......... 79% 36.8M 1s 2024-08-25T19:47:53.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:133950K .......... .......... .......... .......... .......... 79% 15.9M 1s 2024-08-25T19:47:53.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134000K .......... .......... .......... .......... .......... 79% 46.9M 1s 2024-08-25T19:47:53.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134050K .......... .......... .......... .......... .......... 79% 20.4M 1s 2024-08-25T19:47:53.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134100K .......... .......... .......... .......... .......... 79% 40.3M 1s 2024-08-25T19:47:53.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134150K .......... .......... .......... .......... .......... 79% 29.0M 1s 2024-08-25T19:47:53.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134200K .......... .......... .......... .......... .......... 79% 38.0M 1s 2024-08-25T19:47:53.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134250K .......... .......... .......... .......... .......... 79% 13.8M 1s 2024-08-25T19:47:53.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134300K .......... .......... .......... .......... .......... 79% 48.1M 1s 2024-08-25T19:47:53.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134350K .......... .......... .......... .......... .......... 79% 22.5M 1s 2024-08-25T19:47:53.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134400K .......... .......... .......... .......... .......... 79% 23.9M 1s 2024-08-25T19:47:53.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134450K .......... .......... .......... .......... .......... 79% 37.6M 1s 2024-08-25T19:47:53.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134500K .......... .......... .......... .......... .......... 79% 24.9M 1s 2024-08-25T19:47:53.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134550K .......... .......... .......... .......... .......... 79% 41.0M 1s 2024-08-25T19:47:53.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134600K .......... .......... .......... .......... .......... 79% 26.2M 1s 2024-08-25T19:47:53.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134650K .......... .......... .......... .......... .......... 79% 24.3M 1s 2024-08-25T19:47:53.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134700K .......... .......... .......... .......... .......... 79% 20.3M 1s 2024-08-25T19:47:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134750K .......... .......... .......... .......... .......... 79% 25.9M 1s 2024-08-25T19:47:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134800K .......... .......... .......... .......... .......... 79% 33.7M 1s 2024-08-25T19:47:53.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134850K .......... .......... .......... .......... .......... 80% 42.3M 1s 2024-08-25T19:47:53.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134900K .......... .......... .......... .......... .......... 80% 22.7M 1s 2024-08-25T19:47:53.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:134950K .......... .......... .......... .......... .......... 80% 21.9M 1s 2024-08-25T19:47:53.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135000K .......... .......... .......... .......... .......... 80% 15.0M 1s 2024-08-25T19:47:53.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135050K .......... .......... .......... .......... .......... 80% 50.0M 1s 2024-08-25T19:47:53.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135100K .......... .......... .......... .......... .......... 80% 24.1M 1s 2024-08-25T19:47:53.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135150K .......... .......... .......... .......... .......... 80% 22.9M 1s 2024-08-25T19:47:53.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135200K .......... .......... .......... .......... .......... 80% 23.4M 1s 2024-08-25T19:47:53.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135250K .......... .......... .......... .......... .......... 80% 22.7M 1s 2024-08-25T19:47:53.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135300K .......... .......... .......... .......... .......... 80% 22.7M 1s 2024-08-25T19:47:53.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135350K .......... .......... .......... .......... .......... 80% 41.2M 1s 2024-08-25T19:47:53.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135400K .......... .......... .......... .......... .......... 80% 23.8M 1s 2024-08-25T19:47:53.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135450K .......... .......... .......... .......... .......... 80% 28.9M 1s 2024-08-25T19:47:53.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135500K .......... .......... .......... .......... .......... 80% 23.1M 1s 2024-08-25T19:47:53.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135550K .......... .......... .......... .......... .......... 80% 25.5M 1s 2024-08-25T19:47:53.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135600K .......... .......... .......... .......... .......... 80% 21.9M 1s 2024-08-25T19:47:53.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135650K .......... .......... .......... .......... .......... 80% 42.5M 1s 2024-08-25T19:47:53.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135700K .......... .......... .......... .......... .......... 80% 26.9M 1s 2024-08-25T19:47:53.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135750K .......... .......... .......... .......... .......... 80% 23.3M 1s 2024-08-25T19:47:53.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135800K .......... .......... .......... .......... .......... 80% 27.9M 1s 2024-08-25T19:47:53.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135850K .......... .......... .......... .......... .......... 80% 33.3M 1s 2024-08-25T19:47:53.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135900K .......... .......... .......... .......... .......... 80% 22.2M 1s 2024-08-25T19:47:53.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:135950K .......... .......... .......... .......... .......... 80% 23.5M 1s 2024-08-25T19:47:53.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136000K .......... .......... .......... .......... .......... 80% 34.7M 1s 2024-08-25T19:47:53.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136050K .......... .......... .......... .......... .......... 80% 41.2M 1s 2024-08-25T19:47:53.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136100K .......... .......... .......... .......... .......... 80% 17.6M 1s 2024-08-25T19:47:53.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136150K .......... .......... .......... .......... .......... 80% 28.9M 1s 2024-08-25T19:47:53.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136200K .......... .......... .......... .......... .......... 80% 23.3M 1s 2024-08-25T19:47:53.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136250K .......... .......... .......... .......... .......... 80% 49.4M 1s 2024-08-25T19:47:53.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136300K .......... .......... .......... .......... .......... 80% 38.4M 1s 2024-08-25T19:47:53.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136350K .......... .......... .......... .......... .......... 80% 12.5M 1s 2024-08-25T19:47:53.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136400K .......... .......... .......... .......... .......... 80% 54.4M 1s 2024-08-25T19:47:53.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136450K .......... .......... .......... .......... .......... 80% 32.1M 1s 2024-08-25T19:47:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136500K .......... .......... .......... .......... .......... 81% 24.4M 1s 2024-08-25T19:47:53.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136550K .......... .......... .......... .......... .......... 81% 24.3M 1s 2024-08-25T19:47:53.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136600K .......... .......... .......... .......... .......... 81% 42.6M 1s 2024-08-25T19:47:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136650K .......... .......... .......... .......... .......... 81% 20.8M 1s 2024-08-25T19:47:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136700K .......... .......... .......... .......... .......... 81% 25.7M 1s 2024-08-25T19:47:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136750K .......... .......... .......... .......... .......... 81% 11.6M 1s 2024-08-25T19:47:53.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136800K .......... .......... .......... .......... .......... 81% 33.9M 1s 2024-08-25T19:47:53.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136850K .......... .......... .......... .......... .......... 81% 31.8M 1s 2024-08-25T19:47:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136900K .......... .......... .......... .......... .......... 81% 40.4M 1s 2024-08-25T19:47:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:136950K .......... .......... .......... .......... .......... 81% 20.1M 1s 2024-08-25T19:47:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137000K .......... .......... .......... .......... .......... 81% 40.9M 1s 2024-08-25T19:47:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137050K .......... .......... .......... .......... .......... 81% 15.5M 1s 2024-08-25T19:47:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137100K .......... .......... .......... .......... .......... 81% 55.8M 1s 2024-08-25T19:47:53.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137150K .......... .......... .......... .......... .......... 81% 23.7M 1s 2024-08-25T19:47:53.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137200K .......... .......... .......... .......... .......... 81% 16.3M 1s 2024-08-25T19:47:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137250K .......... .......... .......... .......... .......... 81% 21.9M 1s 2024-08-25T19:47:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137300K .......... .......... .......... .......... .......... 81% 36.1M 1s 2024-08-25T19:47:53.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137350K .......... .......... .......... .......... .......... 81% 26.7M 1s 2024-08-25T19:47:53.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137400K .......... .......... .......... .......... .......... 81% 35.2M 1s 2024-08-25T19:47:53.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137450K .......... .......... .......... .......... .......... 81% 29.9M 1s 2024-08-25T19:47:53.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137500K .......... .......... .......... .......... .......... 81% 33.1M 1s 2024-08-25T19:47:53.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137550K .......... .......... .......... .......... .......... 81% 17.1M 1s 2024-08-25T19:47:53.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137600K .......... .......... .......... .......... .......... 81% 21.8M 1s 2024-08-25T19:47:53.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137650K .......... .......... .......... .......... .......... 81% 38.8M 1s 2024-08-25T19:47:53.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137700K .......... .......... .......... .......... .......... 81% 25.3M 1s 2024-08-25T19:47:53.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137750K .......... .......... .......... .......... .......... 81% 42.1M 1s 2024-08-25T19:47:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137800K .......... .......... .......... .......... .......... 81% 22.0M 1s 2024-08-25T19:47:53.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137850K .......... .......... .......... .......... .......... 81% 25.5M 1s 2024-08-25T19:47:53.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137900K .......... .......... .......... .......... .......... 81% 40.8M 1s 2024-08-25T19:47:53.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:137950K .......... .......... .......... .......... .......... 81% 22.7M 1s 2024-08-25T19:47:53.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138000K .......... .......... .......... .......... .......... 81% 27.4M 1s 2024-08-25T19:47:53.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138050K .......... .......... .......... .......... .......... 81% 41.5M 1s 2024-08-25T19:47:53.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138100K .......... .......... .......... .......... .......... 81% 25.4M 1s 2024-08-25T19:47:53.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138150K .......... .......... .......... .......... .......... 81% 29.8M 1s 2024-08-25T19:47:53.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138200K .......... .......... .......... .......... .......... 82% 27.8M 1s 2024-08-25T19:47:53.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138250K .......... .......... .......... .......... .......... 82% 28.6M 1s 2024-08-25T19:47:53.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138300K .......... .......... .......... .......... .......... 82% 24.3M 1s 2024-08-25T19:47:53.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138350K .......... .......... .......... .......... .......... 82% 19.7M 1s 2024-08-25T19:47:53.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138400K .......... .......... .......... .......... .......... 82% 24.9M 1s 2024-08-25T19:47:53.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138450K .......... .......... .......... .......... .......... 82% 41.9M 1s 2024-08-25T19:47:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138500K .......... .......... .......... .......... .......... 82% 25.1M 1s 2024-08-25T19:47:53.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138550K .......... .......... .......... .......... .......... 82% 32.6M 1s 2024-08-25T19:47:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138600K .......... .......... .......... .......... .......... 82% 23.0M 1s 2024-08-25T19:47:53.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138650K .......... .......... .......... .......... .......... 82% 49.6M 1s 2024-08-25T19:47:53.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138700K .......... .......... .......... .......... .......... 82% 23.0M 1s 2024-08-25T19:47:53.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138750K .......... .......... .......... .......... .......... 82% 24.3M 1s 2024-08-25T19:47:53.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138800K .......... .......... .......... .......... .......... 82% 36.5M 1s 2024-08-25T19:47:53.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138850K .......... .......... .......... .......... .......... 82% 23.6M 1s 2024-08-25T19:47:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138900K .......... .......... .......... .......... .......... 82% 36.9M 1s 2024-08-25T19:47:53.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:138950K .......... .......... .......... .......... .......... 82% 23.5M 1s 2024-08-25T19:47:53.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139000K .......... .......... .......... .......... .......... 82% 27.5M 1s 2024-08-25T19:47:53.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139050K .......... .......... .......... .......... .......... 82% 31.3M 1s 2024-08-25T19:47:53.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139100K .......... .......... .......... .......... .......... 82% 48.9M 1s 2024-08-25T19:47:53.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139150K .......... .......... .......... .......... .......... 82% 17.4M 1s 2024-08-25T19:47:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139200K .......... .......... .......... .......... .......... 82% 41.4M 1s 2024-08-25T19:47:53.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139250K .......... .......... .......... .......... .......... 82% 24.5M 1s 2024-08-25T19:47:53.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139300K .......... .......... .......... .......... .......... 82% 44.1M 1s 2024-08-25T19:47:53.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139350K .......... .......... .......... .......... .......... 82% 24.5M 1s 2024-08-25T19:47:53.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139400K .......... .......... .......... .......... .......... 82% 19.2M 1s 2024-08-25T19:47:53.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139450K .......... .......... .......... .......... .......... 82% 21.5M 1s 2024-08-25T19:47:53.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139500K .......... .......... .......... .......... .......... 82% 24.0M 1s 2024-08-25T19:47:53.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139550K .......... .......... .......... .......... .......... 82% 25.5M 1s 2024-08-25T19:47:53.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139600K .......... .......... .......... .......... .......... 82% 42.3M 1s 2024-08-25T19:47:53.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139650K .......... .......... .......... .......... .......... 82% 24.7M 1s 2024-08-25T19:47:53.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139700K .......... .......... .......... .......... .......... 82% 39.3M 1s 2024-08-25T19:47:53.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139750K .......... .......... .......... .......... .......... 82% 25.0M 1s 2024-08-25T19:47:53.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139800K .......... .......... .......... .......... .......... 82% 39.3M 1s 2024-08-25T19:47:53.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139850K .......... .......... .......... .......... .......... 82% 22.7M 1s 2024-08-25T19:47:53.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139900K .......... .......... .......... .......... .......... 83% 47.3M 1s 2024-08-25T19:47:53.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:139950K .......... .......... .......... .......... .......... 83% 23.4M 1s 2024-08-25T19:47:53.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140000K .......... .......... .......... .......... .......... 83% 23.3M 1s 2024-08-25T19:47:53.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140050K .......... .......... .......... .......... .......... 83% 22.3M 1s 2024-08-25T19:47:53.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140100K .......... .......... .......... .......... .......... 83% 41.5M 1s 2024-08-25T19:47:53.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140150K .......... .......... .......... .......... .......... 83% 22.3M 1s 2024-08-25T19:47:53.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140200K .......... .......... .......... .......... .......... 83% 40.1M 1s 2024-08-25T19:47:53.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140250K .......... .......... .......... .......... .......... 83% 34.9M 1s 2024-08-25T19:47:53.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140300K .......... .......... .......... .......... .......... 83% 31.0M 1s 2024-08-25T19:47:53.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140350K .......... .......... .......... .......... .......... 83% 24.3M 1s 2024-08-25T19:47:53.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140400K .......... .......... .......... .......... .......... 83% 25.8M 1s 2024-08-25T19:47:53.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140450K .......... .......... .......... .......... .......... 83% 43.2M 1s 2024-08-25T19:47:53.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140500K .......... .......... .......... .......... .......... 83% 23.3M 1s 2024-08-25T19:47:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140550K .......... .......... .......... .......... .......... 83% 27.1M 1s 2024-08-25T19:47:53.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140600K .......... .......... .......... .......... .......... 83% 30.4M 1s 2024-08-25T19:47:53.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140650K .......... .......... .......... .......... .......... 83% 22.5M 1s 2024-08-25T19:47:53.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140700K .......... .......... .......... .......... .......... 83% 71.7M 1s 2024-08-25T19:47:53.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140750K .......... .......... .......... .......... .......... 83% 19.6M 1s 2024-08-25T19:47:53.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140800K .......... .......... .......... .......... .......... 83% 25.9M 1s 2024-08-25T19:47:53.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140850K .......... .......... .......... .......... .......... 83% 42.9M 1s 2024-08-25T19:47:53.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140900K .......... .......... .......... .......... .......... 83% 25.9M 1s 2024-08-25T19:47:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:140950K .......... .......... .......... .......... .......... 83% 36.4M 1s 2024-08-25T19:47:53.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141000K .......... .......... .......... .......... .......... 83% 25.5M 1s 2024-08-25T19:47:53.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141050K .......... .......... .......... .......... .......... 83% 33.7M 1s 2024-08-25T19:47:53.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141100K .......... .......... .......... .......... .......... 83% 28.1M 1s 2024-08-25T19:47:53.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141150K .......... .......... .......... .......... .......... 83% 23.5M 1s 2024-08-25T19:47:53.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141200K .......... .......... .......... .......... .......... 83% 22.2M 1s 2024-08-25T19:47:53.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141250K .......... .......... .......... .......... .......... 83% 40.7M 1s 2024-08-25T19:47:53.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141300K .......... .......... .......... .......... .......... 83% 26.9M 1s 2024-08-25T19:47:53.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141350K .......... .......... .......... .......... .......... 83% 37.1M 1s 2024-08-25T19:47:53.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141400K .......... .......... .......... .......... .......... 83% 21.3M 1s 2024-08-25T19:47:53.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141450K .......... .......... .......... .......... .......... 83% 39.2M 1s 2024-08-25T19:47:53.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141500K .......... .......... .......... .......... .......... 83% 21.3M 1s 2024-08-25T19:47:53.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141550K .......... .......... .......... .......... .......... 84% 27.6M 1s 2024-08-25T19:47:53.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141600K .......... .......... .......... .......... .......... 84% 37.2M 1s 2024-08-25T19:47:53.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141650K .......... .......... .......... .......... .......... 84% 24.9M 1s 2024-08-25T19:47:53.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141700K .......... .......... .......... .......... .......... 84% 35.3M 1s 2024-08-25T19:47:53.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141750K .......... .......... .......... .......... .......... 84% 26.5M 1s 2024-08-25T19:47:53.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141800K .......... .......... .......... .......... .......... 84% 34.4M 1s 2024-08-25T19:47:53.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141850K .......... .......... .......... .......... .......... 84% 22.6M 1s 2024-08-25T19:47:53.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141900K .......... .......... .......... .......... .......... 84% 26.5M 1s 2024-08-25T19:47:53.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:141950K .......... .......... .......... .......... .......... 84% 36.4M 1s 2024-08-25T19:47:53.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142000K .......... .......... .......... .......... .......... 84% 30.8M 1s 2024-08-25T19:47:53.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142050K .......... .......... .......... .......... .......... 84% 23.3M 1s 2024-08-25T19:47:53.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142100K .......... .......... .......... .......... .......... 84% 37.2M 1s 2024-08-25T19:47:53.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142150K .......... .......... .......... .......... .......... 84% 25.0M 1s 2024-08-25T19:47:53.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142200K .......... .......... .......... .......... .......... 84% 38.4M 1s 2024-08-25T19:47:53.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142250K .......... .......... .......... .......... .......... 84% 23.0M 1s 2024-08-25T19:47:53.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142300K .......... .......... .......... .......... .......... 84% 45.2M 1s 2024-08-25T19:47:53.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142350K .......... .......... .......... .......... .......... 84% 22.0M 1s 2024-08-25T19:47:53.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142400K .......... .......... .......... .......... .......... 84% 23.8M 1s 2024-08-25T19:47:53.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142450K .......... .......... .......... .......... .......... 84% 24.6M 1s 2024-08-25T19:47:53.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142500K .......... .......... .......... .......... .......... 84% 51.4M 1s 2024-08-25T19:47:53.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142550K .......... .......... .......... .......... .......... 84% 24.6M 1s 2024-08-25T19:47:53.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142600K .......... .......... .......... .......... .......... 84% 38.3M 1s 2024-08-25T19:47:53.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142650K .......... .......... .......... .......... .......... 84% 23.2M 1s 2024-08-25T19:47:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142700K .......... .......... .......... .......... .......... 84% 39.4M 1s 2024-08-25T19:47:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142750K .......... .......... .......... .......... .......... 84% 27.1M 1s 2024-08-25T19:47:53.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142800K .......... .......... .......... .......... .......... 84% 23.5M 1s 2024-08-25T19:47:53.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142850K .......... .......... .......... .......... .......... 84% 41.7M 1s 2024-08-25T19:47:53.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142900K .......... .......... .......... .......... .......... 84% 23.5M 1s 2024-08-25T19:47:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:142950K .......... .......... .......... .......... .......... 84% 24.1M 1s 2024-08-25T19:47:53.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143000K .......... .......... .......... .......... .......... 84% 46.7M 1s 2024-08-25T19:47:53.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143050K .......... .......... .......... .......... .......... 84% 24.4M 1s 2024-08-25T19:47:53.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143100K .......... .......... .......... .......... .......... 84% 31.6M 1s 2024-08-25T19:47:53.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143150K .......... .......... .......... .......... .......... 84% 27.4M 1s 2024-08-25T19:47:53.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143200K .......... .......... .......... .......... .......... 84% 25.0M 1s 2024-08-25T19:47:53.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143250K .......... .......... .......... .......... .......... 85% 44.9M 1s 2024-08-25T19:47:53.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143300K .......... .......... .......... .......... .......... 85% 15.8M 1s 2024-08-25T19:47:53.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143350K .......... .......... .......... .......... .......... 85% 14.2M 1s 2024-08-25T19:47:53.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143400K .......... .......... .......... .......... .......... 85% 18.3M 1s 2024-08-25T19:47:53.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143450K .......... .......... .......... .......... .......... 85% 20.1M 1s 2024-08-25T19:47:53.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143500K .......... .......... .......... .......... .......... 85% 48.9M 1s 2024-08-25T19:47:53.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143550K .......... .......... .......... .......... .......... 85% 19.9M 1s 2024-08-25T19:47:53.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143600K .......... .......... .......... .......... .......... 85% 11.3M 1s 2024-08-25T19:47:53.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143650K .......... .......... .......... .......... .......... 85% 33.3M 1s 2024-08-25T19:47:53.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143700K .......... .......... .......... .......... .......... 85% 17.7M 1s 2024-08-25T19:47:53.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143750K .......... .......... .......... .......... .......... 85% 21.1M 1s 2024-08-25T19:47:53.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143800K .......... .......... .......... .......... .......... 85% 6.83M 1s 2024-08-25T19:47:53.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143850K .......... .......... .......... .......... .......... 85% 10.6M 1s 2024-08-25T19:47:53.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143900K .......... .......... .......... .......... .......... 85% 23.1M 1s 2024-08-25T19:47:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:143950K .......... .......... .......... .......... .......... 85% 23.3M 1s 2024-08-25T19:47:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144000K .......... .......... .......... .......... .......... 85% 23.4M 1s 2024-08-25T19:47:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144050K .......... .......... .......... .......... .......... 85% 56.0M 1s 2024-08-25T19:47:53.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144100K .......... .......... .......... .......... .......... 85% 23.6M 1s 2024-08-25T19:47:53.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144150K .......... .......... .......... .......... .......... 85% 37.0M 1s 2024-08-25T19:47:53.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144200K .......... .......... .......... .......... .......... 85% 22.7M 1s 2024-08-25T19:47:53.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144250K .......... .......... .......... .......... .......... 85% 48.3M 1s 2024-08-25T19:47:53.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144300K .......... .......... .......... .......... .......... 85% 24.8M 1s 2024-08-25T19:47:53.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144350K .......... .......... .......... .......... .......... 85% 23.8M 1s 2024-08-25T19:47:53.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144400K .......... .......... .......... .......... .......... 85% 38.9M 1s 2024-08-25T19:47:53.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144450K .......... .......... .......... .......... .......... 85% 23.6M 1s 2024-08-25T19:47:53.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144500K .......... .......... .......... .......... .......... 85% 51.7M 1s 2024-08-25T19:47:53.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144550K .......... .......... .......... .......... .......... 85% 24.5M 1s 2024-08-25T19:47:53.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144600K .......... .......... .......... .......... .......... 85% 35.6M 1s 2024-08-25T19:47:53.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144650K .......... .......... .......... .......... .......... 85% 21.4M 1s 2024-08-25T19:47:53.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144700K .......... .......... .......... .......... .......... 85% 29.4M 1s 2024-08-25T19:47:53.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144750K .......... .......... .......... .......... .......... 85% 22.8M 1s 2024-08-25T19:47:53.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144800K .......... .......... .......... .......... .......... 85% 49.4M 1s 2024-08-25T19:47:53.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144850K .......... .......... .......... .......... .......... 85% 23.5M 1s 2024-08-25T19:47:53.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144900K .......... .......... .......... .......... .......... 85% 42.1M 1s 2024-08-25T19:47:53.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:144950K .......... .......... .......... .......... .......... 86% 22.4M 1s 2024-08-25T19:47:53.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145000K .......... .......... .......... .......... .......... 86% 40.8M 1s 2024-08-25T19:47:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145050K .......... .......... .......... .......... .......... 86% 22.5M 1s 2024-08-25T19:47:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145100K .......... .......... .......... .......... .......... 86% 39.8M 1s 2024-08-25T19:47:53.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145150K .......... .......... .......... .......... .......... 86% 17.3M 1s 2024-08-25T19:47:53.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145200K .......... .......... .......... .......... .......... 86% 41.7M 1s 2024-08-25T19:47:53.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145250K .......... .......... .......... .......... .......... 86% 27.2M 1s 2024-08-25T19:47:53.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145300K .......... .......... .......... .......... .......... 86% 46.3M 1s 2024-08-25T19:47:53.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145350K .......... .......... .......... .......... .......... 86% 21.0M 1s 2024-08-25T19:47:53.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145400K .......... .......... .......... .......... .......... 86% 52.3M 1s 2024-08-25T19:47:53.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145450K .......... .......... .......... .......... .......... 86% 22.3M 1s 2024-08-25T19:47:53.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145500K .......... .......... .......... .......... .......... 86% 39.4M 1s 2024-08-25T19:47:53.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145550K .......... .......... .......... .......... .......... 86% 26.3M 1s 2024-08-25T19:47:53.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145600K .......... .......... .......... .......... .......... 86% 25.1M 1s 2024-08-25T19:47:53.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145650K .......... .......... .......... .......... .......... 86% 23.2M 1s 2024-08-25T19:47:53.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145700K .......... .......... .......... .......... .......... 86% 45.2M 1s 2024-08-25T19:47:53.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145750K .......... .......... .......... .......... .......... 86% 22.4M 1s 2024-08-25T19:47:53.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145800K .......... .......... .......... .......... .......... 86% 38.5M 1s 2024-08-25T19:47:53.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145850K .......... .......... .......... .......... .......... 86% 27.6M 1s 2024-08-25T19:47:53.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145900K .......... .......... .......... .......... .......... 86% 34.1M 1s 2024-08-25T19:47:53.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:145950K .......... .......... .......... .......... .......... 86% 23.7M 1s 2024-08-25T19:47:53.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146000K .......... .......... .......... .......... .......... 86% 23.4M 1s 2024-08-25T19:47:53.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146050K .......... .......... .......... .......... .......... 86% 37.0M 1s 2024-08-25T19:47:53.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146100K .......... .......... .......... .......... .......... 86% 26.7M 1s 2024-08-25T19:47:53.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146150K .......... .......... .......... .......... .......... 86% 24.0M 1s 2024-08-25T19:47:53.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146200K .......... .......... .......... .......... .......... 86% 39.2M 1s 2024-08-25T19:47:53.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146250K .......... .......... .......... .......... .......... 86% 37.3M 1s 2024-08-25T19:47:53.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146300K .......... .......... .......... .......... .......... 86% 23.7M 1s 2024-08-25T19:47:53.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146350K .......... .......... .......... .......... .......... 86% 26.5M 1s 2024-08-25T19:47:53.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146400K .......... .......... .......... .......... .......... 86% 23.4M 1s 2024-08-25T19:47:53.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146450K .......... .......... .......... .......... .......... 86% 31.1M 1s 2024-08-25T19:47:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146500K .......... .......... .......... .......... .......... 86% 28.0M 1s 2024-08-25T19:47:53.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146550K .......... .......... .......... .......... .......... 86% 33.7M 1s 2024-08-25T19:47:53.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146600K .......... .......... .......... .......... .......... 86% 29.7M 1s 2024-08-25T19:47:53.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146650K .......... .......... .......... .......... .......... 87% 37.5M 1s 2024-08-25T19:47:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146700K .......... .......... .......... .......... .......... 87% 24.2M 1s 2024-08-25T19:47:53.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146750K .......... .......... .......... .......... .......... 87% 22.8M 1s 2024-08-25T19:47:53.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146800K .......... .......... .......... .......... .......... 87% 38.1M 1s 2024-08-25T19:47:53.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146850K .......... .......... .......... .......... .......... 87% 26.8M 1s 2024-08-25T19:47:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146900K .......... .......... .......... .......... .......... 87% 20.8M 1s 2024-08-25T19:47:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:146950K .......... .......... .......... .......... .......... 87% 41.6M 1s 2024-08-25T19:47:53.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147000K .......... .......... .......... .......... .......... 87% 26.8M 1s 2024-08-25T19:47:53.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147050K .......... .......... .......... .......... .......... 87% 38.1M 1s 2024-08-25T19:47:53.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147100K .......... .......... .......... .......... .......... 87% 21.8M 1s 2024-08-25T19:47:53.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147150K .......... .......... .......... .......... .......... 87% 27.9M 1s 2024-08-25T19:47:53.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147200K .......... .......... .......... .......... .......... 87% 37.5M 1s 2024-08-25T19:47:53.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147250K .......... .......... .......... .......... .......... 87% 24.0M 1s 2024-08-25T19:47:53.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147300K .......... .......... .......... .......... .......... 87% 48.7M 1s 2024-08-25T19:47:53.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147350K .......... .......... .......... .......... .......... 87% 23.1M 1s 2024-08-25T19:47:53.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147400K .......... .......... .......... .......... .......... 87% 41.1M 1s 2024-08-25T19:47:53.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147450K .......... .......... .......... .......... .......... 87% 23.1M 1s 2024-08-25T19:47:53.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147500K .......... .......... .......... .......... .......... 87% 25.4M 1s 2024-08-25T19:47:53.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147550K .......... .......... .......... .......... .......... 87% 28.1M 1s 2024-08-25T19:47:53.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147600K .......... .......... .......... .......... .......... 87% 38.5M 1s 2024-08-25T19:47:53.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147650K .......... .......... .......... .......... .......... 87% 22.6M 1s 2024-08-25T19:47:53.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147700K .......... .......... .......... .......... .......... 87% 39.9M 1s 2024-08-25T19:47:53.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147750K .......... .......... .......... .......... .......... 87% 24.2M 1s 2024-08-25T19:47:53.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147800K .......... .......... .......... .......... .......... 87% 44.8M 1s 2024-08-25T19:47:53.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147850K .......... .......... .......... .......... .......... 87% 24.3M 1s 2024-08-25T19:47:53.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147900K .......... .......... .......... .......... .......... 87% 41.8M 1s 2024-08-25T19:47:53.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:147950K .......... .......... .......... .......... .......... 87% 8.42M 1s 2024-08-25T19:47:53.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148000K .......... .......... .......... .......... .......... 87% 174M 1s 2024-08-25T19:47:53.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148050K .......... .......... .......... .......... .......... 87% 207M 1s 2024-08-25T19:47:53.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148100K .......... .......... .......... .......... .......... 87% 49.7M 1s 2024-08-25T19:47:53.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148150K .......... .......... .......... .......... .......... 87% 21.7M 1s 2024-08-25T19:47:53.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148200K .......... .......... .......... .......... .......... 87% 32.1M 1s 2024-08-25T19:47:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148250K .......... .......... .......... .......... .......... 87% 24.1M 1s 2024-08-25T19:47:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148300K .......... .......... .......... .......... .......... 88% 55.4M 1s 2024-08-25T19:47:53.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148350K .......... .......... .......... .......... .......... 88% 24.4M 1s 2024-08-25T19:47:53.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148400K .......... .......... .......... .......... .......... 88% 22.3M 1s 2024-08-25T19:47:53.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148450K .......... .......... .......... .......... .......... 88% 41.3M 1s 2024-08-25T19:47:53.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148500K .......... .......... .......... .......... .......... 88% 25.2M 1s 2024-08-25T19:47:53.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148550K .......... .......... .......... .......... .......... 88% 23.3M 1s 2024-08-25T19:47:53.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148600K .......... .......... .......... .......... .......... 88% 43.7M 1s 2024-08-25T19:47:53.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148650K .......... .......... .......... .......... .......... 88% 31.2M 1s 2024-08-25T19:47:53.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148700K .......... .......... .......... .......... .......... 88% 26.9M 1s 2024-08-25T19:47:53.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148750K .......... .......... .......... .......... .......... 88% 21.9M 1s 2024-08-25T19:47:53.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148800K .......... .......... .......... .......... .......... 88% 22.9M 1s 2024-08-25T19:47:53.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148850K .......... .......... .......... .......... .......... 88% 53.5M 1s 2024-08-25T19:47:53.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148900K .......... .......... .......... .......... .......... 88% 24.5M 1s 2024-08-25T19:47:53.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:148950K .......... .......... .......... .......... .......... 88% 30.7M 1s 2024-08-25T19:47:53.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149000K .......... .......... .......... .......... .......... 88% 27.4M 1s 2024-08-25T19:47:53.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149050K .......... .......... .......... .......... .......... 88% 47.3M 1s 2024-08-25T19:47:53.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149100K .......... .......... .......... .......... .......... 88% 24.2M 1s 2024-08-25T19:47:53.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149150K .......... .......... .......... .......... .......... 88% 24.1M 1s 2024-08-25T19:47:53.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149200K .......... .......... .......... .......... .......... 88% 31.9M 1s 2024-08-25T19:47:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149250K .......... .......... .......... .......... .......... 88% 26.8M 1s 2024-08-25T19:47:53.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149300K .......... .......... .......... .......... .......... 88% 23.6M 1s 2024-08-25T19:47:53.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149350K .......... .......... .......... .......... .......... 88% 43.6M 1s 2024-08-25T19:47:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149400K .......... .......... .......... .......... .......... 88% 23.8M 1s 2024-08-25T19:47:53.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149450K .......... .......... .......... .......... .......... 88% 29.1M 1s 2024-08-25T19:47:53.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149500K .......... .......... .......... .......... .......... 88% 29.2M 1s 2024-08-25T19:47:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149550K .......... .......... .......... .......... .......... 88% 25.7M 1s 2024-08-25T19:47:53.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149600K .......... .......... .......... .......... .......... 88% 44.5M 1s 2024-08-25T19:47:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149650K .......... .......... .......... .......... .......... 88% 23.0M 1s 2024-08-25T19:47:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149700K .......... .......... .......... .......... .......... 88% 31.1M 1s 2024-08-25T19:47:53.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149750K .......... .......... .......... .......... .......... 88% 28.2M 1s 2024-08-25T19:47:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149800K .......... .......... .......... .......... .......... 88% 13.0M 1s 2024-08-25T19:47:53.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149850K .......... .......... .......... .......... .......... 88% 17.7M 1s 2024-08-25T19:47:53.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149900K .......... .......... .......... .......... .......... 88% 11.3M 1s 2024-08-25T19:47:53.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:149950K .......... .......... .......... .......... .......... 88% 25.5M 1s 2024-08-25T19:47:53.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150000K .......... .......... .......... .......... .......... 89% 28.9M 1s 2024-08-25T19:47:53.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150050K .......... .......... .......... .......... .......... 89% 24.8M 1s 2024-08-25T19:47:53.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150100K .......... .......... .......... .......... .......... 89% 37.5M 1s 2024-08-25T19:47:53.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150150K .......... .......... .......... .......... .......... 89% 33.1M 1s 2024-08-25T19:47:54.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150200K .......... .......... .......... .......... .......... 89% 26.7M 1s 2024-08-25T19:47:54.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150250K .......... .......... .......... .......... .......... 89% 29.3M 1s 2024-08-25T19:47:54.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150300K .......... .......... .......... .......... .......... 89% 18.6M 1s 2024-08-25T19:47:54.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150350K .......... .......... .......... .......... .......... 89% 13.4M 1s 2024-08-25T19:47:54.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150400K .......... .......... .......... .......... .......... 89% 58.1M 1s 2024-08-25T19:47:54.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150450K .......... .......... .......... .......... .......... 89% 24.6M 1s 2024-08-25T19:47:54.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150500K .......... .......... .......... .......... .......... 89% 37.5M 1s 2024-08-25T19:47:54.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150550K .......... .......... .......... .......... .......... 89% 29.7M 1s 2024-08-25T19:47:54.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150600K .......... .......... .......... .......... .......... 89% 36.0M 1s 2024-08-25T19:47:54.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150650K .......... .......... .......... .......... .......... 89% 28.6M 1s 2024-08-25T19:47:54.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150700K .......... .......... .......... .......... .......... 89% 30.3M 1s 2024-08-25T19:47:54.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150750K .......... .......... .......... .......... .......... 89% 21.5M 1s 2024-08-25T19:47:54.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150800K .......... .......... .......... .......... .......... 89% 26.5M 1s 2024-08-25T19:47:54.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150850K .......... .......... .......... .......... .......... 89% 9.83M 1s 2024-08-25T19:47:54.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150900K .......... .......... .......... .......... .......... 89% 78.8M 1s 2024-08-25T19:47:54.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:150950K .......... .......... .......... .......... .......... 89% 20.9M 1s 2024-08-25T19:47:54.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151000K .......... .......... .......... .......... .......... 89% 37.3M 1s 2024-08-25T19:47:54.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151050K .......... .......... .......... .......... .......... 89% 18.6M 1s 2024-08-25T19:47:54.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151100K .......... .......... .......... .......... .......... 89% 58.1M 1s 2024-08-25T19:47:54.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151150K .......... .......... .......... .......... .......... 89% 21.2M 1s 2024-08-25T19:47:54.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151200K .......... .......... .......... .......... .......... 89% 23.7M 1s 2024-08-25T19:47:54.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151250K .......... .......... .......... .......... .......... 89% 40.1M 1s 2024-08-25T19:47:54.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151300K .......... .......... .......... .......... .......... 89% 25.4M 1s 2024-08-25T19:47:54.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151350K .......... .......... .......... .......... .......... 89% 44.4M 1s 2024-08-25T19:47:54.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151400K .......... .......... .......... .......... .......... 89% 24.3M 1s 2024-08-25T19:47:54.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151450K .......... .......... .......... .......... .......... 89% 28.3M 1s 2024-08-25T19:47:54.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151500K .......... .......... .......... .......... .......... 89% 32.0M 1s 2024-08-25T19:47:54.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151550K .......... .......... .......... .......... .......... 89% 18.1M 1s 2024-08-25T19:47:54.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151600K .......... .......... .......... .......... .......... 89% 29.7M 1s 2024-08-25T19:47:54.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151650K .......... .......... .......... .......... .......... 89% 45.7M 1s 2024-08-25T19:47:54.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151700K .......... .......... .......... .......... .......... 90% 21.9M 1s 2024-08-25T19:47:54.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151750K .......... .......... .......... .......... .......... 90% 42.4M 1s 2024-08-25T19:47:54.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151800K .......... .......... .......... .......... .......... 90% 24.5M 1s 2024-08-25T19:47:54.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151850K .......... .......... .......... .......... .......... 90% 48.5M 1s 2024-08-25T19:47:54.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151900K .......... .......... .......... .......... .......... 90% 21.2M 1s 2024-08-25T19:47:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:151950K .......... .......... .......... .......... .......... 90% 23.7M 1s 2024-08-25T19:47:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152000K .......... .......... .......... .......... .......... 90% 24.3M 1s 2024-08-25T19:47:54.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152050K .......... .......... .......... .......... .......... 90% 40.4M 1s 2024-08-25T19:47:54.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152100K .......... .......... .......... .......... .......... 90% 23.9M 1s 2024-08-25T19:47:54.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152150K .......... .......... .......... .......... .......... 90% 8.96M 1s 2024-08-25T19:47:54.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152200K .......... .......... .......... .......... .......... 90% 11.9M 1s 2024-08-25T19:47:54.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152250K .......... .......... .......... .......... .......... 90% 37.9M 1s 2024-08-25T19:47:54.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152300K .......... .......... .......... .......... .......... 90% 25.2M 1s 2024-08-25T19:47:54.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152350K .......... .......... .......... .......... .......... 90% 22.6M 1s 2024-08-25T19:47:54.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152400K .......... .......... .......... .......... .......... 90% 42.6M 1s 2024-08-25T19:47:54.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152450K .......... .......... .......... .......... .......... 90% 23.8M 1s 2024-08-25T19:47:54.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152500K .......... .......... .......... .......... .......... 90% 21.7M 1s 2024-08-25T19:47:54.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152550K .......... .......... .......... .......... .......... 90% 33.9M 1s 2024-08-25T19:47:54.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152600K .......... .......... .......... .......... .......... 90% 24.5M 1s 2024-08-25T19:47:54.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152650K .......... .......... .......... .......... .......... 90% 71.6M 1s 2024-08-25T19:47:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152700K .......... .......... .......... .......... .......... 90% 20.2M 1s 2024-08-25T19:47:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152750K .......... .......... .......... .......... .......... 90% 28.4M 1s 2024-08-25T19:47:54.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152800K .......... .......... .......... .......... .......... 90% 36.7M 1s 2024-08-25T19:47:54.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152850K .......... .......... .......... .......... .......... 90% 24.4M 1s 2024-08-25T19:47:54.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152900K .......... .......... .......... .......... .......... 90% 43.3M 1s 2024-08-25T19:47:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:152950K .......... .......... .......... .......... .......... 90% 24.5M 1s 2024-08-25T19:47:54.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153000K .......... .......... .......... .......... .......... 90% 37.1M 1s 2024-08-25T19:47:54.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153050K .......... .......... .......... .......... .......... 90% 24.2M 1s 2024-08-25T19:47:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153100K .......... .......... .......... .......... .......... 90% 22.6M 1s 2024-08-25T19:47:54.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153150K .......... .......... .......... .......... .......... 90% 24.8M 1s 2024-08-25T19:47:54.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153200K .......... .......... .......... .......... .......... 90% 38.4M 1s 2024-08-25T19:47:54.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153250K .......... .......... .......... .......... .......... 90% 22.9M 1s 2024-08-25T19:47:54.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153300K .......... .......... .......... .......... .......... 90% 50.8M 1s 2024-08-25T19:47:54.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153350K .......... .......... .......... .......... .......... 91% 22.4M 1s 2024-08-25T19:47:54.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153400K .......... .......... .......... .......... .......... 91% 47.2M 1s 2024-08-25T19:47:54.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153450K .......... .......... .......... .......... .......... 91% 21.6M 1s 2024-08-25T19:47:54.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153500K .......... .......... .......... .......... .......... 91% 47.5M 1s 2024-08-25T19:47:54.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153550K .......... .......... .......... .......... .......... 91% 23.4M 1s 2024-08-25T19:47:54.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153600K .......... .......... .......... .......... .......... 91% 24.5M 1s 2024-08-25T19:47:54.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153650K .......... .......... .......... .......... .......... 91% 24.8M 1s 2024-08-25T19:47:54.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153700K .......... .......... .......... .......... .......... 91% 39.5M 1s 2024-08-25T19:47:54.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153750K .......... .......... .......... .......... .......... 91% 23.3M 1s 2024-08-25T19:47:54.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153800K .......... .......... .......... .......... .......... 91% 40.5M 1s 2024-08-25T19:47:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153850K .......... .......... .......... .......... .......... 91% 22.4M 1s 2024-08-25T19:47:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153900K .......... .......... .......... .......... .......... 91% 55.7M 1s 2024-08-25T19:47:54.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:153950K .......... .......... .......... .......... .......... 91% 24.9M 1s 2024-08-25T19:47:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154000K .......... .......... .......... .......... .......... 91% 23.3M 1s 2024-08-25T19:47:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154050K .......... .......... .......... .......... .......... 91% 39.1M 1s 2024-08-25T19:47:54.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154100K .......... .......... .......... .......... .......... 91% 24.8M 1s 2024-08-25T19:47:54.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154150K .......... .......... .......... .......... .......... 91% 42.8M 1s 2024-08-25T19:47:54.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154200K .......... .......... .......... .......... .......... 91% 23.1M 1s 2024-08-25T19:47:54.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154250K .......... .......... .......... .......... .......... 91% 24.0M 1s 2024-08-25T19:47:54.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154300K .......... .......... .......... .......... .......... 91% 38.0M 1s 2024-08-25T19:47:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154350K .......... .......... .......... .......... .......... 91% 23.6M 1s 2024-08-25T19:47:54.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154400K .......... .......... .......... .......... .......... 91% 26.3M 1s 2024-08-25T19:47:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154450K .......... .......... .......... .......... .......... 91% 42.3M 1s 2024-08-25T19:47:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154500K .......... .......... .......... .......... .......... 91% 24.7M 1s 2024-08-25T19:47:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154550K .......... .......... .......... .......... .......... 91% 40.9M 1s 2024-08-25T19:47:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154600K .......... .......... .......... .......... .......... 91% 23.8M 1s 2024-08-25T19:47:54.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154650K .......... .......... .......... .......... .......... 91% 45.1M 1s 2024-08-25T19:47:54.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154700K .......... .......... .......... .......... .......... 91% 24.9M 1s 2024-08-25T19:47:54.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154750K .......... .......... .......... .......... .......... 91% 23.5M 1s 2024-08-25T19:47:54.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154800K .......... .......... .......... .......... .......... 91% 24.1M 1s 2024-08-25T19:47:54.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154850K .......... .......... .......... .......... .......... 91% 40.8M 1s 2024-08-25T19:47:54.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154900K .......... .......... .......... .......... .......... 91% 23.1M 1s 2024-08-25T19:47:54.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:154950K .......... .......... .......... .......... .......... 91% 43.7M 1s 2024-08-25T19:47:54.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155000K .......... .......... .......... .......... .......... 91% 20.7M 1s 2024-08-25T19:47:54.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155050K .......... .......... .......... .......... .......... 92% 41.9M 0s 2024-08-25T19:47:54.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155100K .......... .......... .......... .......... .......... 92% 24.7M 0s 2024-08-25T19:47:54.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155150K .......... .......... .......... .......... .......... 92% 24.0M 0s 2024-08-25T19:47:54.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155200K .......... .......... .......... .......... .......... 92% 36.5M 0s 2024-08-25T19:47:54.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155250K .......... .......... .......... .......... .......... 92% 24.9M 0s 2024-08-25T19:47:54.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155300K .......... .......... .......... .......... .......... 92% 44.4M 0s 2024-08-25T19:47:54.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155350K .......... .......... .......... .......... .......... 92% 21.6M 0s 2024-08-25T19:47:54.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155400K .......... .......... .......... .......... .......... 92% 24.0M 0s 2024-08-25T19:47:54.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155450K .......... .......... .......... .......... .......... 92% 50.6M 0s 2024-08-25T19:47:54.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155500K .......... .......... .......... .......... .......... 92% 21.3M 0s 2024-08-25T19:47:54.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155550K .......... .......... .......... .......... .......... 92% 25.5M 0s 2024-08-25T19:47:54.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155600K .......... .......... .......... .......... .......... 92% 38.1M 0s 2024-08-25T19:47:54.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155650K .......... .......... .......... .......... .......... 92% 24.2M 0s 2024-08-25T19:47:54.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155700K .......... .......... .......... .......... .......... 92% 47.4M 0s 2024-08-25T19:47:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155750K .......... .......... .......... .......... .......... 92% 22.1M 0s 2024-08-25T19:47:54.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155800K .......... .......... .......... .......... .......... 92% 46.3M 0s 2024-08-25T19:47:54.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155850K .......... .......... .......... .......... .......... 92% 21.8M 0s 2024-08-25T19:47:54.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155900K .......... .......... .......... .......... .......... 92% 26.2M 0s 2024-08-25T19:47:54.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:155950K .......... .......... .......... .......... .......... 92% 22.4M 0s 2024-08-25T19:47:54.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156000K .......... .......... .......... .......... .......... 92% 50.2M 0s 2024-08-25T19:47:54.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156050K .......... .......... .......... .......... .......... 92% 25.4M 0s 2024-08-25T19:47:54.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156100K .......... .......... .......... .......... .......... 92% 37.2M 0s 2024-08-25T19:47:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156150K .......... .......... .......... .......... .......... 92% 21.7M 0s 2024-08-25T19:47:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156200K .......... .......... .......... .......... .......... 92% 44.9M 0s 2024-08-25T19:47:54.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156250K .......... .......... .......... .......... .......... 92% 24.5M 0s 2024-08-25T19:47:54.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156300K .......... .......... .......... .......... .......... 92% 42.3M 0s 2024-08-25T19:47:54.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156350K .......... .......... .......... .......... .......... 92% 21.3M 0s 2024-08-25T19:47:54.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156400K .......... .......... .......... .......... .......... 92% 25.4M 0s 2024-08-25T19:47:54.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156450K .......... .......... .......... .......... .......... 92% 50.0M 0s 2024-08-25T19:47:54.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156500K .......... .......... .......... .......... .......... 92% 14.2M 0s 2024-08-25T19:47:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156550K .......... .......... .......... .......... .......... 92% 12.2M 0s 2024-08-25T19:47:54.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156600K .......... .......... .......... .......... .......... 92% 13.1M 0s 2024-08-25T19:47:54.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156650K .......... .......... .......... .......... .......... 92% 19.4M 0s 2024-08-25T19:47:54.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156700K .......... .......... .......... .......... .......... 92% 43.8M 0s 2024-08-25T19:47:54.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156750K .......... .......... .......... .......... .......... 93% 17.6M 0s 2024-08-25T19:47:54.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156800K .......... .......... .......... .......... .......... 93% 16.2M 0s 2024-08-25T19:47:54.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156850K .......... .......... .......... .......... .......... 93% 38.5M 0s 2024-08-25T19:47:54.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156900K .......... .......... .......... .......... .......... 93% 24.5M 0s 2024-08-25T19:47:54.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:156950K .......... .......... .......... .......... .......... 93% 48.0M 0s 2024-08-25T19:47:54.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157000K .......... .......... .......... .......... .......... 93% 24.1M 0s 2024-08-25T19:47:54.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157050K .......... .......... .......... .......... .......... 93% 30.9M 0s 2024-08-25T19:47:54.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157100K .......... .......... .......... .......... .......... 93% 26.1M 0s 2024-08-25T19:47:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157150K .......... .......... .......... .......... .......... 93% 24.5M 0s 2024-08-25T19:47:54.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157200K .......... .......... .......... .......... .......... 93% 24.0M 0s 2024-08-25T19:47:54.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157250K .......... .......... .......... .......... .......... 93% 53.2M 0s 2024-08-25T19:47:54.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157300K .......... .......... .......... .......... .......... 93% 24.5M 0s 2024-08-25T19:47:54.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157350K .......... .......... .......... .......... .......... 93% 38.1M 0s 2024-08-25T19:47:54.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157400K .......... .......... .......... .......... .......... 93% 23.2M 0s 2024-08-25T19:47:54.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157450K .......... .......... .......... .......... .......... 93% 47.6M 0s 2024-08-25T19:47:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157500K .......... .......... .......... .......... .......... 93% 25.3M 0s 2024-08-25T19:47:54.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157550K .......... .......... .......... .......... .......... 93% 23.8M 0s 2024-08-25T19:47:54.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157600K .......... .......... .......... .......... .......... 93% 38.9M 0s 2024-08-25T19:47:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157650K .......... .......... .......... .......... .......... 93% 21.0M 0s 2024-08-25T19:47:54.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157700K .......... .......... .......... .......... .......... 93% 31.4M 0s 2024-08-25T19:47:54.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157750K .......... .......... .......... .......... .......... 93% 40.7M 0s 2024-08-25T19:47:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157800K .......... .......... .......... .......... .......... 93% 21.9M 0s 2024-08-25T19:47:54.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157850K .......... .......... .......... .......... .......... 93% 38.6M 0s 2024-08-25T19:47:54.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157900K .......... .......... .......... .......... .......... 93% 12.7M 0s 2024-08-25T19:47:54.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:157950K .......... .......... .......... .......... .......... 93% 22.3M 0s 2024-08-25T19:47:54.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158000K .......... .......... .......... .......... .......... 93% 45.0M 0s 2024-08-25T19:47:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158050K .......... .......... .......... .......... .......... 93% 6.80M 0s 2024-08-25T19:47:54.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158100K .......... .......... .......... .......... .......... 93% 11.2M 0s 2024-08-25T19:47:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158150K .......... .......... .......... .......... .......... 93% 28.3M 0s 2024-08-25T19:47:54.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158200K .......... .......... .......... .......... .......... 93% 22.1M 0s 2024-08-25T19:47:54.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158250K .......... .......... .......... .......... .......... 93% 38.4M 0s 2024-08-25T19:47:54.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158300K .......... .......... .......... .......... .......... 93% 25.4M 0s 2024-08-25T19:47:54.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158350K .......... .......... .......... .......... .......... 93% 34.9M 0s 2024-08-25T19:47:54.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158400K .......... .......... .......... .......... .......... 93% 30.2M 0s 2024-08-25T19:47:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158450K .......... .......... .......... .......... .......... 94% 25.2M 0s 2024-08-25T19:47:54.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158500K .......... .......... .......... .......... .......... 94% 44.0M 0s 2024-08-25T19:47:54.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158550K .......... .......... .......... .......... .......... 94% 23.5M 0s 2024-08-25T19:47:54.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158600K .......... .......... .......... .......... .......... 94% 37.2M 0s 2024-08-25T19:47:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158650K .......... .......... .......... .......... .......... 94% 24.7M 0s 2024-08-25T19:47:54.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158700K .......... .......... .......... .......... .......... 94% 47.9M 0s 2024-08-25T19:47:54.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158750K .......... .......... .......... .......... .......... 94% 15.0M 0s 2024-08-25T19:47:54.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158800K .......... .......... .......... .......... .......... 94% 40.3M 0s 2024-08-25T19:47:54.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158850K .......... .......... .......... .......... .......... 94% 26.1M 0s 2024-08-25T19:47:54.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158900K .......... .......... .......... .......... .......... 94% 36.0M 0s 2024-08-25T19:47:54.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:158950K .......... .......... .......... .......... .......... 94% 24.2M 0s 2024-08-25T19:47:54.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159000K .......... .......... .......... .......... .......... 94% 40.8M 0s 2024-08-25T19:47:54.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159050K .......... .......... .......... .......... .......... 94% 27.9M 0s 2024-08-25T19:47:54.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159100K .......... .......... .......... .......... .......... 94% 39.8M 0s 2024-08-25T19:47:54.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159150K .......... .......... .......... .......... .......... 94% 22.1M 0s 2024-08-25T19:47:54.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159200K .......... .......... .......... .......... .......... 94% 25.1M 0s 2024-08-25T19:47:54.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159250K .......... .......... .......... .......... .......... 94% 37.7M 0s 2024-08-25T19:47:54.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159300K .......... .......... .......... .......... .......... 94% 27.2M 0s 2024-08-25T19:47:54.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159350K .......... .......... .......... .......... .......... 94% 23.8M 0s 2024-08-25T19:47:54.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159400K .......... .......... .......... .......... .......... 94% 37.1M 0s 2024-08-25T19:47:54.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159450K .......... .......... .......... .......... .......... 94% 23.9M 0s 2024-08-25T19:47:54.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159500K .......... .......... .......... .......... .......... 94% 43.6M 0s 2024-08-25T19:47:54.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159550K .......... .......... .......... .......... .......... 94% 23.7M 0s 2024-08-25T19:47:54.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159600K .......... .......... .......... .......... .......... 94% 25.7M 0s 2024-08-25T19:47:54.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159650K .......... .......... .......... .......... .......... 94% 38.5M 0s 2024-08-25T19:47:54.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159700K .......... .......... .......... .......... .......... 94% 23.4M 0s 2024-08-25T19:47:54.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159750K .......... .......... .......... .......... .......... 94% 37.1M 0s 2024-08-25T19:47:54.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159800K .......... .......... .......... .......... .......... 94% 27.1M 0s 2024-08-25T19:47:54.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159850K .......... .......... .......... .......... .......... 94% 24.5M 0s 2024-08-25T19:47:54.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159900K .......... .......... .......... .......... .......... 94% 48.6M 0s 2024-08-25T19:47:54.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:159950K .......... .......... .......... .......... .......... 94% 19.9M 0s 2024-08-25T19:47:54.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160000K .......... .......... .......... .......... .......... 94% 25.2M 0s 2024-08-25T19:47:54.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160050K .......... .......... .......... .......... .......... 94% 46.7M 0s 2024-08-25T19:47:54.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160100K .......... .......... .......... .......... .......... 95% 24.6M 0s 2024-08-25T19:47:54.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160150K .......... .......... .......... .......... .......... 95% 33.3M 0s 2024-08-25T19:47:54.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160200K .......... .......... .......... .......... .......... 95% 27.7M 0s 2024-08-25T19:47:54.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160250K .......... .......... .......... .......... .......... 95% 45.4M 0s 2024-08-25T19:47:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160300K .......... .......... .......... .......... .......... 95% 24.9M 0s 2024-08-25T19:47:54.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160350K .......... .......... .......... .......... .......... 95% 20.9M 0s 2024-08-25T19:47:54.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160400K .......... .......... .......... .......... .......... 95% 27.7M 0s 2024-08-25T19:47:54.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160450K .......... .......... .......... .......... .......... 95% 34.0M 0s 2024-08-25T19:47:54.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160500K .......... .......... .......... .......... .......... 95% 25.3M 0s 2024-08-25T19:47:54.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160550K .......... .......... .......... .......... .......... 95% 42.8M 0s 2024-08-25T19:47:54.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160600K .......... .......... .......... .......... .......... 95% 26.7M 0s 2024-08-25T19:47:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160650K .......... .......... .......... .......... .......... 95% 32.3M 0s 2024-08-25T19:47:54.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160700K .......... .......... .......... .......... .......... 95% 26.7M 0s 2024-08-25T19:47:54.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160750K .......... .......... .......... .......... .......... 95% 21.9M 0s 2024-08-25T19:47:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160800K .......... .......... .......... .......... .......... 95% 40.6M 0s 2024-08-25T19:47:54.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160850K .......... .......... .......... .......... .......... 95% 27.9M 0s 2024-08-25T19:47:54.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160900K .......... .......... .......... .......... .......... 95% 25.9M 0s 2024-08-25T19:47:54.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:160950K .......... .......... .......... .......... .......... 95% 33.8M 0s 2024-08-25T19:47:54.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161000K .......... .......... .......... .......... .......... 95% 24.2M 0s 2024-08-25T19:47:54.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161050K .......... .......... .......... .......... .......... 95% 41.8M 0s 2024-08-25T19:47:54.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161100K .......... .......... .......... .......... .......... 95% 22.8M 0s 2024-08-25T19:47:54.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161150K .......... .......... .......... .......... .......... 95% 26.9M 0s 2024-08-25T19:47:54.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161200K .......... .......... .......... .......... .......... 95% 39.0M 0s 2024-08-25T19:47:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161250K .......... .......... .......... .......... .......... 95% 22.6M 0s 2024-08-25T19:47:54.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161300K .......... .......... .......... .......... .......... 95% 45.1M 0s 2024-08-25T19:47:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161350K .......... .......... .......... .......... .......... 95% 24.1M 0s 2024-08-25T19:47:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161400K .......... .......... .......... .......... .......... 95% 30.1M 0s 2024-08-25T19:47:54.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161450K .......... .......... .......... .......... .......... 95% 28.0M 0s 2024-08-25T19:47:54.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161500K .......... .......... .......... .......... .......... 95% 49.2M 0s 2024-08-25T19:47:54.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161550K .......... .......... .......... .......... .......... 95% 15.3M 0s 2024-08-25T19:47:54.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161600K .......... .......... .......... .......... .......... 95% 55.5M 0s 2024-08-25T19:47:54.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161650K .......... .......... .......... .......... .......... 95% 26.9M 0s 2024-08-25T19:47:54.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161700K .......... .......... .......... .......... .......... 95% 27.7M 0s 2024-08-25T19:47:54.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161750K .......... .......... .......... .......... .......... 95% 37.0M 0s 2024-08-25T19:47:54.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161800K .......... .......... .......... .......... .......... 96% 29.9M 0s 2024-08-25T19:47:54.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161850K .......... .......... .......... .......... .......... 96% 21.4M 0s 2024-08-25T19:47:54.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161900K .......... .......... .......... .......... .......... 96% 38.2M 0s 2024-08-25T19:47:54.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:161950K .......... .......... .......... .......... .......... 96% 24.9M 0s 2024-08-25T19:47:54.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162000K .......... .......... .......... .......... .......... 96% 22.0M 0s 2024-08-25T19:47:54.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162050K .......... .......... .......... .......... .......... 96% 46.3M 0s 2024-08-25T19:47:54.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162100K .......... .......... .......... .......... .......... 96% 26.2M 0s 2024-08-25T19:47:54.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162150K .......... .......... .......... .......... .......... 96% 34.6M 0s 2024-08-25T19:47:54.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162200K .......... .......... .......... .......... .......... 96% 34.3M 0s 2024-08-25T19:47:54.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162250K .......... .......... .......... .......... .......... 96% 19.6M 0s 2024-08-25T19:47:54.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162300K .......... .......... .......... .......... .......... 96% 44.1M 0s 2024-08-25T19:47:54.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162350K .......... .......... .......... .......... .......... 96% 24.6M 0s 2024-08-25T19:47:54.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162400K .......... .......... .......... .......... .......... 96% 24.1M 0s 2024-08-25T19:47:54.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162450K .......... .......... .......... .......... .......... 96% 31.6M 0s 2024-08-25T19:47:54.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162500K .......... .......... .......... .......... .......... 96% 27.6M 0s 2024-08-25T19:47:54.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162550K .......... .......... .......... .......... .......... 96% 46.0M 0s 2024-08-25T19:47:54.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162600K .......... .......... .......... .......... .......... 96% 29.8M 0s 2024-08-25T19:47:54.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162650K .......... .......... .......... .......... .......... 96% 31.2M 0s 2024-08-25T19:47:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162700K .......... .......... .......... .......... .......... 96% 24.1M 0s 2024-08-25T19:47:54.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162750K .......... .......... .......... .......... .......... 96% 33.0M 0s 2024-08-25T19:47:54.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162800K .......... .......... .......... .......... .......... 96% 6.18M 0s 2024-08-25T19:47:54.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162850K .......... .......... .......... .......... .......... 96% 12.5M 0s 2024-08-25T19:47:54.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162900K .......... .......... .......... .......... .......... 96% 29.0M 0s 2024-08-25T19:47:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:162950K .......... .......... .......... .......... .......... 96% 25.8M 0s 2024-08-25T19:47:54.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163000K .......... .......... .......... .......... .......... 96% 26.4M 0s 2024-08-25T19:47:54.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163050K .......... .......... .......... .......... .......... 96% 35.5M 0s 2024-08-25T19:47:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163100K .......... .......... .......... .......... .......... 96% 26.3M 0s 2024-08-25T19:47:54.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163150K .......... .......... .......... .......... .......... 96% 25.6M 0s 2024-08-25T19:47:54.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163200K .......... .......... .......... .......... .......... 96% 40.8M 0s 2024-08-25T19:47:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163250K .......... .......... .......... .......... .......... 96% 22.8M 0s 2024-08-25T19:47:54.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163300K .......... .......... .......... .......... .......... 96% 24.1M 0s 2024-08-25T19:47:54.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163350K .......... .......... .......... .......... .......... 96% 50.5M 0s 2024-08-25T19:47:54.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163400K .......... .......... .......... .......... .......... 96% 21.2M 0s 2024-08-25T19:47:54.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163450K .......... .......... .......... .......... .......... 96% 34.4M 0s 2024-08-25T19:47:54.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163500K .......... .......... .......... .......... .......... 97% 26.9M 0s 2024-08-25T19:47:54.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163550K .......... .......... .......... .......... .......... 97% 23.4M 0s 2024-08-25T19:47:54.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163600K .......... .......... .......... .......... .......... 97% 45.1M 0s 2024-08-25T19:47:54.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163650K .......... .......... .......... .......... .......... 97% 24.7M 0s 2024-08-25T19:47:54.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163700K .......... .......... .......... .......... .......... 97% 38.2M 0s 2024-08-25T19:47:54.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163750K .......... .......... .......... .......... .......... 97% 24.3M 0s 2024-08-25T19:47:54.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163800K .......... .......... .......... .......... .......... 97% 42.1M 0s 2024-08-25T19:47:54.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163850K .......... .......... .......... .......... .......... 97% 24.8M 0s 2024-08-25T19:47:54.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163900K .......... .......... .......... .......... .......... 97% 36.7M 0s 2024-08-25T19:47:54.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:163950K .......... .......... .......... .......... .......... 97% 17.6M 0s 2024-08-25T19:47:54.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164000K .......... .......... .......... .......... .......... 97% 32.6M 0s 2024-08-25T19:47:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164050K .......... .......... .......... .......... .......... 97% 27.0M 0s 2024-08-25T19:47:54.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164100K .......... .......... .......... .......... .......... 97% 48.6M 0s 2024-08-25T19:47:54.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164150K .......... .......... .......... .......... .......... 97% 24.3M 0s 2024-08-25T19:47:54.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164200K .......... .......... .......... .......... .......... 97% 40.3M 0s 2024-08-25T19:47:54.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164250K .......... .......... .......... .......... .......... 97% 27.3M 0s 2024-08-25T19:47:54.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164300K .......... .......... .......... .......... .......... 97% 20.1M 0s 2024-08-25T19:47:54.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164350K .......... .......... .......... .......... .......... 97% 25.4M 0s 2024-08-25T19:47:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164400K .......... .......... .......... .......... .......... 97% 11.6M 0s 2024-08-25T19:47:54.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164450K .......... .......... .......... .......... .......... 97% 12.1M 0s 2024-08-25T19:47:54.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164500K .......... .......... .......... .......... .......... 97% 38.8M 0s 2024-08-25T19:47:54.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164550K .......... .......... .......... .......... .......... 97% 25.1M 0s 2024-08-25T19:47:54.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164600K .......... .......... .......... .......... .......... 97% 18.0M 0s 2024-08-25T19:47:54.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164650K .......... .......... .......... .......... .......... 97% 22.8M 0s 2024-08-25T19:47:54.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164700K .......... .......... .......... .......... .......... 97% 39.3M 0s 2024-08-25T19:47:54.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164750K .......... .......... .......... .......... .......... 97% 18.7M 0s 2024-08-25T19:47:54.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164800K .......... .......... .......... .......... .......... 97% 21.2M 0s 2024-08-25T19:47:54.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164850K .......... .......... .......... .......... .......... 97% 44.8M 0s 2024-08-25T19:47:54.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164900K .......... .......... .......... .......... .......... 97% 10.9M 0s 2024-08-25T19:47:54.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:164950K .......... .......... .......... .......... .......... 97% 28.1M 0s 2024-08-25T19:47:54.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165000K .......... .......... .......... .......... .......... 97% 29.6M 0s 2024-08-25T19:47:54.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165050K .......... .......... .......... .......... .......... 97% 25.2M 0s 2024-08-25T19:47:54.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165100K .......... .......... .......... .......... .......... 97% 42.8M 0s 2024-08-25T19:47:54.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165150K .......... .......... .......... .......... .......... 98% 26.6M 0s 2024-08-25T19:47:54.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165200K .......... .......... .......... .......... .......... 98% 20.5M 0s 2024-08-25T19:47:54.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165250K .......... .......... .......... .......... .......... 98% 45.5M 0s 2024-08-25T19:47:54.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165300K .......... .......... .......... .......... .......... 98% 31.6M 0s 2024-08-25T19:47:54.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165350K .......... .......... .......... .......... .......... 98% 28.7M 0s 2024-08-25T19:47:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165400K .......... .......... .......... .......... .......... 98% 26.1M 0s 2024-08-25T19:47:54.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165450K .......... .......... .......... .......... .......... 98% 34.7M 0s 2024-08-25T19:47:54.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165500K .......... .......... .......... .......... .......... 98% 25.7M 0s 2024-08-25T19:47:54.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165550K .......... .......... .......... .......... .......... 98% 23.7M 0s 2024-08-25T19:47:54.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165600K .......... .......... .......... .......... .......... 98% 23.3M 0s 2024-08-25T19:47:54.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165650K .......... .......... .......... .......... .......... 98% 47.3M 0s 2024-08-25T19:47:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165700K .......... .......... .......... .......... .......... 98% 23.5M 0s 2024-08-25T19:47:54.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165750K .......... .......... .......... .......... .......... 98% 40.0M 0s 2024-08-25T19:47:54.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165800K .......... .......... .......... .......... .......... 98% 22.8M 0s 2024-08-25T19:47:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165850K .......... .......... .......... .......... .......... 98% 43.4M 0s 2024-08-25T19:47:54.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165900K .......... .......... .......... .......... .......... 98% 25.8M 0s 2024-08-25T19:47:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:165950K .......... .......... .......... .......... .......... 98% 23.6M 0s 2024-08-25T19:47:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166000K .......... .......... .......... .......... .......... 98% 36.7M 0s 2024-08-25T19:47:54.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166050K .......... .......... .......... .......... .......... 98% 25.6M 0s 2024-08-25T19:47:54.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166100K .......... .......... .......... .......... .......... 98% 23.1M 0s 2024-08-25T19:47:54.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166150K .......... .......... .......... .......... .......... 98% 34.5M 0s 2024-08-25T19:47:54.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166200K .......... .......... .......... .......... .......... 98% 30.1M 0s 2024-08-25T19:47:54.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166250K .......... .......... .......... .......... .......... 98% 35.6M 0s 2024-08-25T19:47:54.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166300K .......... .......... .......... .......... .......... 98% 24.9M 0s 2024-08-25T19:47:54.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166350K .......... .......... .......... .......... .......... 98% 23.7M 0s 2024-08-25T19:47:54.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166400K .......... .......... .......... .......... .......... 98% 41.3M 0s 2024-08-25T19:47:54.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166450K .......... .......... .......... .......... .......... 98% 24.6M 0s 2024-08-25T19:47:54.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166500K .......... .......... .......... .......... .......... 98% 27.6M 0s 2024-08-25T19:47:54.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166550K .......... .......... .......... .......... .......... 98% 33.1M 0s 2024-08-25T19:47:54.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166600K .......... .......... .......... .......... .......... 98% 43.6M 0s 2024-08-25T19:47:54.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166650K .......... .......... .......... .......... .......... 98% 23.1M 0s 2024-08-25T19:47:54.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166700K .......... .......... .......... .......... .......... 98% 25.8M 0s 2024-08-25T19:47:54.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166750K .......... .......... .......... .......... .......... 98% 21.7M 0s 2024-08-25T19:47:54.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166800K .......... .......... .......... .......... .......... 98% 31.1M 0s 2024-08-25T19:47:54.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166850K .......... .......... .......... .......... .......... 99% 29.1M 0s 2024-08-25T19:47:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166900K .......... .......... .......... .......... .......... 99% 34.9M 0s 2024-08-25T19:47:54.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:166950K .......... .......... .......... .......... .......... 99% 26.8M 0s 2024-08-25T19:47:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167000K .......... .......... .......... .......... .......... 99% 38.9M 0s 2024-08-25T19:47:54.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167050K .......... .......... .......... .......... .......... 99% 25.7M 0s 2024-08-25T19:47:54.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167100K .......... .......... .......... .......... .......... 99% 43.9M 0s 2024-08-25T19:47:54.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167150K .......... .......... .......... .......... .......... 99% 21.8M 0s 2024-08-25T19:47:54.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167200K .......... .......... .......... .......... .......... 99% 24.7M 0s 2024-08-25T19:47:54.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167250K .......... .......... .......... .......... .......... 99% 24.7M 0s 2024-08-25T19:47:54.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167300K .......... .......... .......... .......... .......... 99% 37.4M 0s 2024-08-25T19:47:54.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167350K .......... .......... .......... .......... .......... 99% 23.9M 0s 2024-08-25T19:47:54.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167400K .......... .......... .......... .......... .......... 99% 51.3M 0s 2024-08-25T19:47:54.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167450K .......... .......... .......... .......... .......... 99% 25.4M 0s 2024-08-25T19:47:54.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167500K .......... .......... .......... .......... .......... 99% 32.3M 0s 2024-08-25T19:47:54.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167550K .......... .......... .......... .......... .......... 99% 19.6M 0s 2024-08-25T19:47:54.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167600K .......... .......... .......... .......... .......... 99% 36.9M 0s 2024-08-25T19:47:54.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167650K .......... .......... .......... .......... .......... 99% 36.9M 0s 2024-08-25T19:47:54.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167700K .......... .......... .......... .......... .......... 99% 26.3M 0s 2024-08-25T19:47:54.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167750K .......... .......... .......... .......... .......... 99% 40.1M 0s 2024-08-25T19:47:54.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167800K .......... .......... .......... .......... .......... 99% 26.2M 0s 2024-08-25T19:47:54.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167850K .......... .......... .......... .......... .......... 99% 20.9M 0s 2024-08-25T19:47:54.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167900K .......... .......... .......... .......... .......... 99% 42.6M 0s 2024-08-25T19:47:54.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:167950K .......... .......... .......... .......... .......... 99% 26.4M 0s 2024-08-25T19:47:54.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:168000K .......... .......... .......... .......... .......... 99% 23.4M 0s 2024-08-25T19:47:54.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:168050K .......... .......... .......... .......... .......... 99% 40.0M 0s 2024-08-25T19:47:54.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:168100K .......... .......... .......... .......... .......... 99% 27.0M 0s 2024-08-25T19:47:54.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:168150K .......... .......... .......... .......... .......... 99% 35.6M 0s 2024-08-25T19:47:54.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:168200K .......... .......... .......... .......... .......... 99% 24.6M 0s 2024-08-25T19:47:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:168250K .......... .......... .......... .......... .......... 99% 32.9M 0s 2024-08-25T19:47:54.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:168300K .......... .......... .......... .......... .......... 99% 25.4M 0s 2024-08-25T19:47:54.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:168350K .......... .......... .......... .......... .......... 99% 26.2M 0s 2024-08-25T19:47:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:168400K .......... .......... .......... .......... .......... 99% 22.4M 0s 2024-08-25T19:47:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:168450K .......... .......... .......... .......... .......... 99% 58.9M 0s 2024-08-25T19:47:54.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:168500K .......... .......... .......... .......... .......... 99% 25.4M 0s 2024-08-25T19:47:54.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:168550K .......... .......... . 100% 61.4M=6.2s 2024-08-25T19:47:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2024-08-25T19:47:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:2024-08-25 19:47:54 (26.4 MB/s) - ‘linux-5.4.tar.gz’ saved [172616875/172616875] 2024-08-25T19:47:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: 2024-08-25T19:47:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..10} 2024-08-25T19:47:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS=() 2024-08-25T19:47:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS=() 2024-08-25T19:47:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS=() 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS=() 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RET=0 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary1 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:47:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:54.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-08-25T19:47:54.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:47:54.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:54.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:47:54.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:47:54.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-08-25T19:47:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:47:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:47:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-08-25T19:47:55.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:47:55.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-08-25T19:47:55.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary1': 2024-08-25T19:47:55.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:47:55.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:47:55.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:47:55.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 10857eeb26cf 2024-08-25T19:47:55.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.10857eeb26cf 2024-08-25T19:47:55.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:47:55.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:47:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:47:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:47:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:47:54 2024 2024-08-25T19:47:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:47:54 2024 2024-08-25T19:47:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:47:54 2024 2024-08-25T19:47:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:47:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:47:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 271f3fac-9544-4101-b3db-355632166a97 2024-08-25T19:47:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:47:55.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:47:55.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:47:55.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-08-25T19:47:55.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd0 2024-08-25T19:47:55.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:47:55.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-08-25T19:47:55.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:47:55.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:47:55.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:47:55.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: cb101061-e9d1-44de-b223-2cc277c3cc08 2024-08-25T19:47:55.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:47:55.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:47:55.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:47:55.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:47:55.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:47:55.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:47:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:47:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:47:55.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary1 2024-08-25T19:47:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-08-25T19:47:55.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:47:55.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary1 2024-08-25T19:47:55.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:47:55.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-08-25T19:47:55.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:55.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:55.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:47:55.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:47:55.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:47:55.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:55.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:55.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:47:55.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:47:55.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:47:55.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:47:55.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary1 2024-08-25T19:47:55.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T19:47:55.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary2 2024-08-25T19:47:55.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-08-25T19:47:55.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary1 2024-08-25T19:47:55.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:55.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-08-25T19:47:55.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:55.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:55.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:47:55.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:47:55.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:47:55.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:55.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-08-25T19:47:56.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:47:56.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:56.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:47:56.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:47:56.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-08-25T19:47:56.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:56.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:56.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:47:56.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:47:56.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-08-25T19:47:56.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 4 2024-08-25T19:47:56.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:47:56.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:47:56.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-08-25T19:47:57.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:47:57.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-08-25T19:47:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary2': 2024-08-25T19:47:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:47:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:47:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:47:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 10958e717592 2024-08-25T19:47:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.10958e717592 2024-08-25T19:47:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:47:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:47:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:47:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:47:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:47:56 2024 2024-08-25T19:47:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:47:56 2024 2024-08-25T19:47:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:47:56 2024 2024-08-25T19:47:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:47:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:47:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: bbfc30c2-dc11-4de7-ada3-fc4241ba8134 2024-08-25T19:47:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:47:57.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:47:57.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:47:57.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-08-25T19:47:57.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd1 2024-08-25T19:47:57.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:47:57.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-08-25T19:47:57.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:47:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:47:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:47:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 0dc48df5-04e9-4b0f-a923-607446060e2c 2024-08-25T19:47:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:47:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:47:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:47:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:47:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:47:57.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:47:57.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:47:57.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:47:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary2 2024-08-25T19:47:57.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-08-25T19:47:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:47:57.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary2 2024-08-25T19:47:57.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T19:47:57.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary3 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:57.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:57.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:57.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:47:57.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:57.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:47:57.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:47:57.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:47:57.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:57.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary2 2024-08-25T19:47:57.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-08-25T19:47:57.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary2 2024-08-25T19:47:57.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-08-25T19:47:57.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:47:57.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:57.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:47:57.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:47:57.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-08-25T19:47:57.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:57.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:57.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:47:57.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:47:57.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-08-25T19:47:57.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:47:57.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-08-25T19:47:58.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:47:58.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 6 2024-08-25T19:47:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:47:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-08-25T19:47:58.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary3': 2024-08-25T19:47:58.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:47:58.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 10a4b3e14c64 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.10a4b3e14c64 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:47:57 2024 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:47:57 2024 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:47:57 2024 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: b1eaf95b-5076-4fa7-856b-29a8cc179965 2024-08-25T19:47:58.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:47:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:47:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:47:58.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-08-25T19:47:58.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd2 2024-08-25T19:47:58.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:47:58.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-08-25T19:47:58.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:47:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:47:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:47:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: a4783edf-4d39-45ca-b2f1-600eeda1ebd0 2024-08-25T19:47:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:47:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:47:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:47:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:47:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:47:58.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:47:58.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:47:58.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:47:58.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary3 2024-08-25T19:47:58.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-08-25T19:47:58.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:47:58.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary3 2024-08-25T19:47:58.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:47:58.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-08-25T19:47:58.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:58.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:58.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:47:58.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:47:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:47:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:47:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:47:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:47:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:47:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T19:47:58.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary4 2024-08-25T19:47:58.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary3 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:58.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:58.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary3 2024-08-25T19:47:58.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:58.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:47:58.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:58.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:47:58.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:47:58.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-08-25T19:47:58.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:47:58.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:58.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-08-25T19:47:58.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:47:58.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:58.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:47:58.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:47:58.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-08-25T19:47:58.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:58.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:58.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:47:58.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:47:58.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-08-25T19:47:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:47:58.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-08-25T19:47:59.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:47:59.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:47:59.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:59.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:59.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:47:59.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:47:59.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:47:59.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 8 2024-08-25T19:47:59.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:47:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-08-25T19:47:59.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary4': 2024-08-25T19:47:59.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:47:59.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:47:59.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:47:59.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 10be80579b4d 2024-08-25T19:47:59.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.10be80579b4d 2024-08-25T19:47:59.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:47:59.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:47:59.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:47:59.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:47:59.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:47:58 2024 2024-08-25T19:47:59.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:47:58 2024 2024-08-25T19:47:59.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:47:58 2024 2024-08-25T19:47:59.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:47:59.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:47:59.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: b75218c8-fbe6-4268-8a2c-89c6f9d197fa 2024-08-25T19:47:59.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:47:59.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:47:59.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:47:59.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-08-25T19:47:59.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd3 2024-08-25T19:47:59.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:47:59.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-08-25T19:47:59.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:47:59.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:47:59.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:47:59.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: aaaa420d-914e-4eca-8010-003b61564c45 2024-08-25T19:47:59.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:47:59.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:47:59.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:47:59.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:47:59.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:47:59.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:47:59.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:47:59.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:47:59.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary4 2024-08-25T19:47:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-08-25T19:47:59.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:47:59.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary4 2024-08-25T19:47:59.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:47:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T19:47:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary5 2024-08-25T19:47:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-08-25T19:47:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary4 2024-08-25T19:47:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:47:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:47:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:47:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary4 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:47:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-08-25T19:47:59.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-08-25T19:47:59.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:47:59.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:59.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:47:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:47:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-08-25T19:47:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:47:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:47:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:47:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:47:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-08-25T19:47:59.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:47:59.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-08-25T19:48:00.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:48:00.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 10 2024-08-25T19:48:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 11 2024-08-25T19:48:00.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-08-25T19:48:00.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:00.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary5': 2024-08-25T19:48:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:48:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:48:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:48:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 10d0627c267 2024-08-25T19:48:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.10d0627c267 2024-08-25T19:48:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:48:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:48:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:48:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:48:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:47:59 2024 2024-08-25T19:48:00.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:47:59 2024 2024-08-25T19:48:00.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:47:59 2024 2024-08-25T19:48:00.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:48:00.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:48:00.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: af8e54d2-12d3-45f1-8e69-422f660f56dc 2024-08-25T19:48:00.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:48:00.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:48:00.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:48:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-08-25T19:48:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd4 2024-08-25T19:48:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:48:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-08-25T19:48:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:48:00.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:48:00.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:48:00.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: e3e89125-618b-4b58-877b-53c376e54c34 2024-08-25T19:48:00.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:48:00.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:48:00.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:48:00.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:48:00.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:48:00.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:48:00.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:48:00.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:48:00.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary5 2024-08-25T19:48:00.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-08-25T19:48:00.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:48:00.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary5 2024-08-25T19:48:00.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:48:00.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-08-25T19:48:00.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:00.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:00.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:00.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary6 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary5 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:00.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:00.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:48:00.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary5 2024-08-25T19:48:00.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:48:00.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:48:00.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:00.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-08-25T19:48:00.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-08-25T19:48:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:48:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:48:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:48:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-08-25T19:48:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:48:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-08-25T19:48:00.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:48:00.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-08-25T19:48:01.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:01.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:01.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:01.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:01.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:01.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:01.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:48:01.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 13 2024-08-25T19:48:01.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-08-25T19:48:01.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:01.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary6': 2024-08-25T19:48:01.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:48:01.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:48:01.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:48:01.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 10df7d05aa48 2024-08-25T19:48:01.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.10df7d05aa48 2024-08-25T19:48:01.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:48:01.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:48:01.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:48:01.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:48:01.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:48:00 2024 2024-08-25T19:48:01.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:48:00 2024 2024-08-25T19:48:01.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:48:00 2024 2024-08-25T19:48:01.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:48:01.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:48:01.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 5641da01-7c6f-4b5d-9e77-32496e6199e6 2024-08-25T19:48:01.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:48:01.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:48:01.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:48:01.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-08-25T19:48:01.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd5 2024-08-25T19:48:01.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:48:01.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-08-25T19:48:01.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:48:01.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:48:01.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:48:01.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 6165615d-01e4-4978-aa28-4cd1a95f1ea7 2024-08-25T19:48:01.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:48:01.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:48:01.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:48:01.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:48:01.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:48:01.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:48:01.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:48:01.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:48:01.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary6 2024-08-25T19:48:01.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-08-25T19:48:01.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:48:01.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary6 2024-08-25T19:48:01.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:48:01.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-08-25T19:48:01.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:01.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:01.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:01.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:01.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary7 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary6 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:01.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary6 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-08-25T19:48:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:01.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:48:01.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:48:01.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:48:01.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:01.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-08-25T19:48:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:48:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:48:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:48:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-08-25T19:48:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:48:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-08-25T19:48:01.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:48:01.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-08-25T19:48:02.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:02.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:02.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:02.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:02.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:02.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:02.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:48:02.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 16 2024-08-25T19:48:02.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 15 2024-08-25T19:48:02.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-08-25T19:48:02.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:02.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:02.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary7': 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 10f112ae7ab8 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.10f112ae7ab8 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:48:01 2024 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:48:01 2024 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:48:01 2024 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:48:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:48:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 540d8938-7a8c-477e-8fc3-e65c72e78c93 2024-08-25T19:48:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:48:02.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:48:02.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:48:02.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-08-25T19:48:02.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd6 2024-08-25T19:48:02.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:48:02.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-08-25T19:48:02.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:48:02.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:48:02.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:48:02.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 5eaf0884-402d-4df7-b2a3-5b13c252ec32 2024-08-25T19:48:02.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:48:02.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:48:02.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:48:02.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:48:02.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:48:02.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:48:02.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary7 2024-08-25T19:48:02.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-08-25T19:48:02.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:48:02.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary7 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:48:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary7 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary8 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary7 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:48:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-08-25T19:48:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:48:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:48:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:48:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-08-25T19:48:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:48:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:48:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:48:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-08-25T19:48:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:48:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-08-25T19:48:02.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:48:02.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-08-25T19:48:03.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:03.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:48:03.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:03.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:03.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:48:03.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:48:03.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:03.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:03.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:48:03.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:03.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:03.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:03.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:03.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 18 2024-08-25T19:48:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 19 2024-08-25T19:48:03.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-08-25T19:48:03.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:03.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary8': 2024-08-25T19:48:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:48:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:48:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:48:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1103fcf7be92 2024-08-25T19:48:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1103fcf7be92 2024-08-25T19:48:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:48:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:48:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:48:03.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:48:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:48:02 2024 2024-08-25T19:48:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:48:02 2024 2024-08-25T19:48:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:48:02 2024 2024-08-25T19:48:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:48:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:48:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 5fdd517d-317f-4c25-b8f4-f5395bd911d6 2024-08-25T19:48:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:48:03.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:48:03.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:48:03.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-08-25T19:48:03.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd7 2024-08-25T19:48:03.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:48:03.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-08-25T19:48:03.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:48:03.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:48:03.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:48:03.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 9bc1c63d-0671-43c9-b4d9-9cb02c91521f 2024-08-25T19:48:03.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:48:03.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:48:03.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:48:03.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:48:03.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:48:03.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:48:03.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:48:03.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:48:03.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary8 2024-08-25T19:48:03.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-08-25T19:48:03.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:48:03.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary8 2024-08-25T19:48:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:48:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-08-25T19:48:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary9 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary8 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:48:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary8 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:03.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-08-25T19:48:03.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:48:03.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:03.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:48:03.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:48:03.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-08-25T19:48:03.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:03.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:03.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:03.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:48:03.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-08-25T19:48:03.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:48:03.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-08-25T19:48:04.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:48:04.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 23 2024-08-25T19:48:04.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 21 2024-08-25T19:48:04.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 22 2024-08-25T19:48:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:04.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-08-25T19:48:04.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:04.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:04.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary9': 2024-08-25T19:48:04.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:48:04.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:48:04.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:48:04.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1118ca57c444 2024-08-25T19:48:04.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1118ca57c444 2024-08-25T19:48:04.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:48:04.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:48:04.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:48:04.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:48:04.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:48:03 2024 2024-08-25T19:48:04.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:48:03 2024 2024-08-25T19:48:04.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:48:03 2024 2024-08-25T19:48:04.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:48:04.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:48:04.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 73264916-cfd5-40d0-83ae-ff7d92659663 2024-08-25T19:48:04.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:48:04.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:48:04.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:48:04.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-08-25T19:48:04.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:04.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:04.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:04.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:04.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:04.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:04.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd8 2024-08-25T19:48:04.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:48:04.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-08-25T19:48:04.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:48:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:48:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:48:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 15f6f4fe-464a-4096-b9b0-4461ac40b03b 2024-08-25T19:48:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:48:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:48:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:48:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:48:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:48:04.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:48:04.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:48:04.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:48:04.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary9 2024-08-25T19:48:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-08-25T19:48:04.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:48:04.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary9 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:48:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary10 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary9 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary9 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:48:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-08-25T19:48:04.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:48:04.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:04.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:48:04.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:48:04.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-08-25T19:48:04.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:04.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:04.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:04.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:48:04.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-08-25T19:48:04.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:48:04.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-08-25T19:48:05.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:48:05.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:05.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:05.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:05.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:48:05.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:05.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:05.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:05.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:05.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:05.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:05.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:05.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:48:05.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 26 2024-08-25T19:48:05.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 25 2024-08-25T19:48:05.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:05.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-08-25T19:48:05.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:05.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary10': 2024-08-25T19:48:05.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 11295834d4be 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.11295834d4be 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:48:04 2024 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:48:04 2024 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:48:04 2024 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:48:05.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:48:05.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: fad93bc2-e996-49cb-a574-393a1cbbe56a 2024-08-25T19:48:05.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:48:05.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:48:05.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:48:05.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-08-25T19:48:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd9 2024-08-25T19:48:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:48:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-08-25T19:48:05.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:48:05.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:48:05.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:48:05.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 9c71dc14-e5bd-44d4-bbf5-b80d694e3eff 2024-08-25T19:48:05.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:48:05.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:48:05.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:48:05.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:48:05.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:48:05.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:48:05.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:48:05.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:48:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary10 2024-08-25T19:48:05.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-08-25T19:48:05.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:48:05.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary10 2024-08-25T19:48:05.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:48:05.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-08-25T19:48:05.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:05.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:05.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:05.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:05.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:48:05.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:05.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:48:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:48:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:48:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 23225 2024-08-25T19:48:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary10 2024-08-25T19:48:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary10 2024-08-25T19:48:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-08-25T19:48:05.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:48:05.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-08-25T19:48:06.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 28 2024-08-25T19:48:06.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 30 2024-08-25T19:48:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 29 2024-08-25T19:48:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:06.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:06.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:06.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:06.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:06.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:06.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:07.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:07.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:07.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:07.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:07.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:07.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:07.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 32 2024-08-25T19:48:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:48:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:48:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:48:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 31 2024-08-25T19:48:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:07.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:07.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:07.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:07.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:48:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:48:08.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:08.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:08.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:08.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:08.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:08.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:08.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 33 2024-08-25T19:48:08.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 34 2024-08-25T19:48:08.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 35 2024-08-25T19:48:08.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:08.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:08.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:09.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:09.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:48:09.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:09.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:09.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:09.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:48:09.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 37 2024-08-25T19:48:09.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:09.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:48:09.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:09.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:09.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:09.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:48:09.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 36 2024-08-25T19:48:09.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:09.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:09.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:09.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:09.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:09.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:09.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:09.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:10.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:10.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:10.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:10.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:10.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:10.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:10.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:10.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:10.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:10.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:10.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:10.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:10.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 40 2024-08-25T19:48:10.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 38 2024-08-25T19:48:10.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 39 2024-08-25T19:48:10.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:10.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:10.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:11.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:11.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:11.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:11.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:11.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:11.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:11.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:11.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:11.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:11.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:11.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:11.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:48:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:48:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:48:11.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 41 2024-08-25T19:48:11.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 42 2024-08-25T19:48:11.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:11.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:12.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:12.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:12.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:12.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:12.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:12.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:48:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:48:12.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 43 2024-08-25T19:48:12.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 44 2024-08-25T19:48:12.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:12.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 45 2024-08-25T19:48:12.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:12.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:13.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:13.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:48:13.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:13.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:13.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:13.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:48:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:48:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:48:13.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:13.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:13.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:13.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:13.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:13.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:13.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 46 2024-08-25T19:48:13.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 47 2024-08-25T19:48:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:13.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:14.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:14.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:14.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:14.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 50 2024-08-25T19:48:14.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 48 2024-08-25T19:48:14.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 49 2024-08-25T19:48:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:15.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:15.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 52 2024-08-25T19:48:15.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:15.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:15.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:15.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:15.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:15.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:15.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 51 2024-08-25T19:48:15.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:15.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:48:15.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:15.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:15.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:48:15.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:48:15.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:15.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:48:16.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:16.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:16.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:16.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:16.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:16.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 54 2024-08-25T19:48:16.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 53 2024-08-25T19:48:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:16.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:17.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:17.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:48:17.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:17.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:17.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:17.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:48:17.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:17.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:48:17.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:17.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:17.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:17.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:48:17.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:17.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:17.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:17.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:17.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:17.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:17.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 56 2024-08-25T19:48:17.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 57 2024-08-25T19:48:17.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:17.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:18.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 55 2024-08-25T19:48:18.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:18.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:18.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:18.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:18.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:18.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:18.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:18.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 59 2024-08-25T19:48:18.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:18.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:18.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:18.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:18.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:18.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:18.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:18.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 58 2024-08-25T19:48:18.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:19.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 61 2024-08-25T19:48:19.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:19.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:19.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:19.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:19.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:19.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:19.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:19.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:19.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:19.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:19.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:19.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:19.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:20.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 60 2024-08-25T19:48:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:20.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 63 2024-08-25T19:48:20.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:20.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:20.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:20.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:20.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:20.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:20.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:20.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:20.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:48:20.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:20.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:20.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:20.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:48:21.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 62 2024-08-25T19:48:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:21.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:48:21.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:21.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:21.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:48:21.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:48:21.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:48:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:48:21.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:21.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:48:21.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:21.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:21.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:21.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:48:21.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 66 2024-08-25T19:48:21.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:22.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 64 2024-08-25T19:48:22.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:22.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:22.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:22.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:22.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:22.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:22.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:22.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 69 2024-08-25T19:48:22.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 68 2024-08-25T19:48:22.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:23.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 65 2024-08-25T19:48:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:23.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 67 2024-08-25T19:48:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:24.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:24.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:24.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:48:24.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:24.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:24.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:24.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:48:24.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 71 2024-08-25T19:48:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:25.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 70 2024-08-25T19:48:25.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:25.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:25.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:25.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:25.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:25.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:25.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:25.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:25.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:48:25.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:25.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:25.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:48:25.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:48:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:48:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 73 2024-08-25T19:48:25.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:25.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 72 2024-08-25T19:48:25.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:26.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 75 2024-08-25T19:48:26.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:26.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:26.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:26.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:26.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:26.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:26.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:26.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 74 2024-08-25T19:48:26.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:27.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 78 2024-08-25T19:48:27.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:27.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:27.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:48:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:48:27.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:27.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:27.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:27.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:27.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:27.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:27.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 77 2024-08-25T19:48:27.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 76 2024-08-25T19:48:27.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:27.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:27.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 79 2024-08-25T19:48:28.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:28.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:28.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:28.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:28.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:28.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:28.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:28.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:28.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:28.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:28.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:28.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:28.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:28.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 81 2024-08-25T19:48:28.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:28.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:28.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:28.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:28.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:28.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:28.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:29.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:29.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:29.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:29.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:29.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:29.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:29.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 80 2024-08-25T19:48:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:48:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:48:29.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:30.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 83 2024-08-25T19:48:30.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:30.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:30.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:30.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:30.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:30.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:30.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:30.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 82 2024-08-25T19:48:30.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:30.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:48:30.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:30.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:30.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:48:30.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:48:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:30.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:30.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:48:30.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:30.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:30.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:30.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:48:31.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:31.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:48:31.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:31.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:31.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:31.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:48:31.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 85 2024-08-25T19:48:31.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:31.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 84 2024-08-25T19:48:31.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:32.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 88 2024-08-25T19:48:32.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:32.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 87 2024-08-25T19:48:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:33.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:33.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:33.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:33.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:33.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:33.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:33.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:33.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:33.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:33.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:33.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:33.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:34.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 91 2024-08-25T19:48:34.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:48:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:48:34.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 86 2024-08-25T19:48:34.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 90 2024-08-25T19:48:34.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:48:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:48:35.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:35.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:48:35.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:35.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:35.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:48:35.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:48:35.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 93 2024-08-25T19:48:35.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:35.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 92 2024-08-25T19:48:35.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:36.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 95 2024-08-25T19:48:36.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:36.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 94 2024-08-25T19:48:36.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 89 2024-08-25T19:48:37.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:37.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:37.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:37.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:37.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:37.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:37.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:37.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:37.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 97 2024-08-25T19:48:38.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 96 2024-08-25T19:48:38.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:38.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:38.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:38.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:38.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:38.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:38.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:38.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:38.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:38.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:38.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:38.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:38.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:38.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:38.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:38.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:38.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:38.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:38.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:38.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:39.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 100 2024-08-25T19:48:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 99 2024-08-25T19:48:39.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:39.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:39.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:39.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:39.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:39.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:39.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:39.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:40.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 98 2024-08-25T19:48:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:40.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:40.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:48:40.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:40.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:40.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:40.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:48:40.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:40.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:48:40.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:40.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:40.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:40.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:48:40.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 102 2024-08-25T19:48:40.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:40.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 101 2024-08-25T19:48:40.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:40.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 105 2024-08-25T19:48:41.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 104 2024-08-25T19:48:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:41.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:41.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:41.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:48:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:48:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:48:42.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 103 2024-08-25T19:48:42.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 106 2024-08-25T19:48:42.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 107 2024-08-25T19:48:42.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:42.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:43.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:43.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:43.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:43.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:43.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:43.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:43.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:43.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:44.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:44.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:48:44.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:44.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:44.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:44.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:48:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:48:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:48:44.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 110 2024-08-25T19:48:44.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:45.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 108 2024-08-25T19:48:45.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 109 2024-08-25T19:48:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:45.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:45.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:45.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:45.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:45.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:48:45.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:45.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:45.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:48:45.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:48:45.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:45.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:48:45.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:45.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:45.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:45.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:48:45.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 112 2024-08-25T19:48:45.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 111 2024-08-25T19:48:45.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:46.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 115 2024-08-25T19:48:46.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:47.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 113 2024-08-25T19:48:47.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 114 2024-08-25T19:48:47.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:47.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:47.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:47.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:47.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:47.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:47.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:47.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:48.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 116 2024-08-25T19:48:48.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 117 2024-08-25T19:48:48.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:48.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:48.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:48.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:48.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:48.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:48.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:48.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:48.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:48.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:48.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:48.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:48.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:48.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:48.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:48.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:48.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:48.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:48.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:48.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:49.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:49.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:49.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:49.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:49.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:49.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:49.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 120 2024-08-25T19:48:49.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 119 2024-08-25T19:48:49.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:49.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:50.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 118 2024-08-25T19:48:50.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:50.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:48:50.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:50.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:50.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:50.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:48:50.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:50.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:50.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:48:50.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:50.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:50.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:50.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:48:50.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 122 2024-08-25T19:48:50.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:50.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 121 2024-08-25T19:48:50.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:50.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 124 2024-08-25T19:48:50.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 125 2024-08-25T19:48:51.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:51.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:51.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:51.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:48:51.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:51.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:51.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:48:51.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:48:51.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:51.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:48:51.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 123 2024-08-25T19:48:51.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:51.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 127 2024-08-25T19:48:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 126 2024-08-25T19:48:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:52.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:52.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:52.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:52.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:52.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:52.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:52.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:52.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:53.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:53.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:53.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:53.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:53.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:53.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:53.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:53.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:53.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:53.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:53.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:53.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:54.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:54.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:48:54.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:54.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:54.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:48:54.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:48:54.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:54.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:48:54.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:54.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:54.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:48:54.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:48:54.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 130 2024-08-25T19:48:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:54.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:54.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:54.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:54.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:54.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:54.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:55.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:55.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:48:55.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:55.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:55.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:48:55.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:48:55.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:55.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:48:55.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:55.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:55.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:48:55.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:48:55.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 129 2024-08-25T19:48:55.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 128 2024-08-25T19:48:55.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:55.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:55.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 132 2024-08-25T19:48:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:55.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 131 2024-08-25T19:48:55.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:56.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 135 2024-08-25T19:48:56.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:56.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 134 2024-08-25T19:48:56.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 133 2024-08-25T19:48:57.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:57.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:57.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:57.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:48:57.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:57.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:57.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:48:57.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:48:57.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 137 2024-08-25T19:48:57.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 136 2024-08-25T19:48:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:57.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:48:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:48:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:48:58.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:58.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:48:58.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:58.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:58.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:48:58.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:48:58.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:58.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:48:58.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:58.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:58.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:48:58.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:48:58.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 139 2024-08-25T19:48:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:48:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:48:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:48:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 140 2024-08-25T19:48:59.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:59.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:48:59.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:48:59.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:48:59.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:48:59.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:48:59.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:48:59.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:48:59.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 138 2024-08-25T19:48:59.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:00.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:00.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:49:00.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:00.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:00.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:49:00.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:49:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 142 2024-08-25T19:49:00.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:00.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:49:00.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:00.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:49:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:49:00.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:00.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 141 2024-08-25T19:49:00.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:49:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:49:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:49:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 145 2024-08-25T19:49:01.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 144 2024-08-25T19:49:01.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:01.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:49:01.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:01.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:01.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:49:01.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:49:01.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:01.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:01.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 143 2024-08-25T19:49:01.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:01.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 147 2024-08-25T19:49:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 146 2024-08-25T19:49:02.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:49:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:49:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:49:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:49:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:49:02.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:49:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:49:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:49:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:49:03.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:03.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:49:03.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:03.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:03.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:49:03.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:49:03.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:03.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:49:03.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:03.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:03.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:49:03.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:49:04.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:04.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:49:04.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:04.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:04.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:49:04.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:49:04.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:04.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:49:04.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:04.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:04.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:49:04.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:49:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 150 2024-08-25T19:49:04.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:04.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 149 2024-08-25T19:49:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:49:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:49:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:49:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 148 2024-08-25T19:49:05.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:05.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:05.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:05.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:49:05.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:05.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:05.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:49:05.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:49:05.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:05.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:49:05.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:05.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:05.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:49:05.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:49:05.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 152 2024-08-25T19:49:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:05.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 151 2024-08-25T19:49:06.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:06.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 155 2024-08-25T19:49:06.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:06.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 153 2024-08-25T19:49:06.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 154 2024-08-25T19:49:07.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:07.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:07.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:07.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:49:07.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:07.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:07.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:49:07.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:49:07.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 156 2024-08-25T19:49:07.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 157 2024-08-25T19:49:08.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:08.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:49:08.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:08.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:08.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:49:08.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:49:08.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:08.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:08.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:49:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:49:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:49:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:08.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:08.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:49:08.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:08.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:08.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:49:08.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:49:09.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:09.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:49:09.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:09.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:09.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:49:09.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:49:09.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 159 2024-08-25T19:49:09.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 160 2024-08-25T19:49:09.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:09.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:49:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:49:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:49:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:10.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 158 2024-08-25T19:49:10.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:10.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:49:10.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:10.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:10.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:49:10.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:49:10.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:10.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:49:10.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:10.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:10.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:49:10.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:49:10.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:10.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 162 2024-08-25T19:49:10.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:10.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 161 2024-08-25T19:49:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:10.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 165 2024-08-25T19:49:11.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 164 2024-08-25T19:49:11.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:11.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:11.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:11.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:49:11.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:11.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:11.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:49:11.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:49:11.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:11.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:49:11.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:11.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:11.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:49:11.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:49:12.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 163 2024-08-25T19:49:12.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:12.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 167 2024-08-25T19:49:12.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:12.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:49:12.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:49:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:49:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:49:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:49:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:49:12.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 166 2024-08-25T19:49:12.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:49:13.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:13.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:13.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:49:13.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:49:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:49:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:13.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:13.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:49:13.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:49:13.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:13.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:49:13.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:13.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:13.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:49:13.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:49:14.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:14.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:14.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:49:14.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:49:14.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:14.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:14.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:14.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:14.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:49:14.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:49:14.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:49:14.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:49:14.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 170 2024-08-25T19:49:14.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:15.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 168 2024-08-25T19:49:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:49:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:49:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:49:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 169 2024-08-25T19:49:15.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:15.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:49:15.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:15.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:15.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:49:15.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:49:15.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:15.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:49:15.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:15.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:15.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:49:15.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:49:15.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:15.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:15.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 172 2024-08-25T19:49:15.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:15.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 171 2024-08-25T19:49:16.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 175 2024-08-25T19:49:16.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:17.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 173 2024-08-25T19:49:17.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 174 2024-08-25T19:49:17.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:17.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:49:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:49:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:49:18.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 177 2024-08-25T19:49:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 176 2024-08-25T19:49:18.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:49:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:18.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:49:18.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:49:18.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:18.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:49:18.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:18.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:18.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:49:18.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:49:18.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:18.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:49:18.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:18.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:18.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:49:18.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:49:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:49:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:49:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:49:19.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:19.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:49:19.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:19.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:19.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:49:19.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:49:19.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 179 2024-08-25T19:49:19.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 180 2024-08-25T19:49:19.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:19.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:20.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 178 2024-08-25T19:49:20.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:20.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:49:20.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:20.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:20.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:49:20.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:49:20.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:20.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:20.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:49:20.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:20.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:20.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:49:20.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:49:20.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 182 2024-08-25T19:49:20.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:20.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 181 2024-08-25T19:49:20.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:20.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 184 2024-08-25T19:49:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:21.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 185 2024-08-25T19:49:21.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:49:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:49:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:49:21.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:21.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:49:21.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:21.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:21.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:49:21.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:49:21.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 183 2024-08-25T19:49:21.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:21.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 186 2024-08-25T19:49:21.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 187 2024-08-25T19:49:21.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:21.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:22.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:22.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:49:22.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:22.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:22.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:49:22.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:49:22.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:22.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:49:22.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:22.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:22.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:49:22.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:49:23.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:23.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:49:23.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:23.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:23.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:49:23.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:49:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:49:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:49:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:49:23.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:23.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:49:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:49:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:49:24.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:24.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:49:24.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:24.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:24.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:49:24.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:49:24.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:24.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:49:24.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:24.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:24.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:49:24.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:49:24.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 190 2024-08-25T19:49:24.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:49:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:24.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:49:24.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:49:24.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:24.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:49:24.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:24.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:24.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:49:24.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:49:25.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:25.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:49:25.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:25.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:25.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:49:25.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:49:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 189 2024-08-25T19:49:25.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 188 2024-08-25T19:49:25.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 192 2024-08-25T19:49:25.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:25.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 191 2024-08-25T19:49:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:26.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 195 2024-08-25T19:49:26.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:26.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 194 2024-08-25T19:49:26.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 193 2024-08-25T19:49:26.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:26.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:27.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:27.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:49:27.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:27.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:27.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:49:27.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:49:27.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 196 2024-08-25T19:49:27.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 197 2024-08-25T19:49:27.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:27.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:49:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:28.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:28.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:49:28.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:49:28.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:28.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:49:28.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:28.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:28.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:49:28.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:49:28.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:28.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:49:28.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:28.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:28.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:49:28.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:49:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:49:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:49:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:49:29.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 200 2024-08-25T19:49:29.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 199 2024-08-25T19:49:29.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:29.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:29.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:29.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:49:29.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:29.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:29.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:49:29.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:49:29.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 198 2024-08-25T19:49:29.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:29.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:49:29.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:29.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:49:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:49:29.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:30.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:30.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:49:30.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:30.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:30.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:49:30.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:49:30.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 202 2024-08-25T19:49:30.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:30.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 201 2024-08-25T19:49:30.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:30.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:49:30.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:30.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:30.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:49:30.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:49:30.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:30.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:30.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:49:30.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:30.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:30.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:49:30.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:49:30.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 205 2024-08-25T19:49:31.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 204 2024-08-25T19:49:31.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:31.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 203 2024-08-25T19:49:31.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:32.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 206 2024-08-25T19:49:32.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 207 2024-08-25T19:49:32.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:32.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:49:32.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:32.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:32.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:49:32.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:49:32.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:32.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:32.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:49:32.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:32.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:32.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:49:32.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:49:32.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:33.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:33.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:49:33.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:33.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:33.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:49:33.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:49:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:49:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:49:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:49:33.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:33.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:49:33.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:33.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:33.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:49:33.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:49:34.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:34.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:49:34.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:34.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:34.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:49:34.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:49:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:49:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:49:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:49:34.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 210 2024-08-25T19:49:34.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:34.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 208 2024-08-25T19:49:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 209 2024-08-25T19:49:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:49:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:49:34.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:49:35.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:49:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:49:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:49:35.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:49:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:49:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:49:35.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 212 2024-08-25T19:49:35.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:35.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 211 2024-08-25T19:49:35.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:36.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 215 2024-08-25T19:49:36.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:49:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:49:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:49:37.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 214 2024-08-25T19:49:37.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 213 2024-08-25T19:49:37.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:37.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:38.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 216 2024-08-25T19:49:38.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 217 2024-08-25T19:49:38.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:38.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:38.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:49:38.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:38.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:38.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:49:38.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:49:38.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:38.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:38.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:49:38.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:38.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:38.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:49:38.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:49:38.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:38.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:49:38.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:38.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:38.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:49:38.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:49:39.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:39.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:49:39.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:39.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:39.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:49:39.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:49:39.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:39.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:49:39.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:39.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:39.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:49:39.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:49:39.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 219 2024-08-25T19:49:39.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 220 2024-08-25T19:49:39.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:39.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:40.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 218 2024-08-25T19:49:40.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:40.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 222 2024-08-25T19:49:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:40.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:40.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:49:40.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:40.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:40.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:49:40.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:49:40.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:40.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:49:40.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:40.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:40.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:49:40.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:49:40.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 221 2024-08-25T19:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:41.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 225 2024-08-25T19:49:41.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 224 2024-08-25T19:49:41.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:41.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:49:41.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:41.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:41.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:49:41.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:49:41.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:49:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:49:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:49:42.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 223 2024-08-25T19:49:42.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:42.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 226 2024-08-25T19:49:42.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 227 2024-08-25T19:49:42.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:42.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:49:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:49:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:49:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:49:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:42.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:49:42.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:49:43.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:43.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:49:43.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:43.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:43.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:49:43.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:49:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:49:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:49:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:49:44.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:44.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:49:44.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:44.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:44.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:49:44.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:49:44.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:44.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:49:44.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:44.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:44.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:49:44.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:49:44.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:44.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:49:44.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:44.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:44.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:49:44.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:49:44.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 230 2024-08-25T19:49:44.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:45.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 228 2024-08-25T19:49:45.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:45.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:49:45.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:49:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:49:45.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 229 2024-08-25T19:49:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:49:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:45.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:49:45.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:49:45.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:45.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:45.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:49:45.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:45.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:45.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:49:45.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:49:45.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:45.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 232 2024-08-25T19:49:45.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 231 2024-08-25T19:49:46.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:46.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 235 2024-08-25T19:49:46.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 233 2024-08-25T19:49:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 234 2024-08-25T19:49:47.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:47.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:47.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:47.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:49:47.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:47.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:47.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:49:47.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:49:48.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 237 2024-08-25T19:49:48.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 236 2024-08-25T19:49:48.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:48.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:49:48.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:48.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:48.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:49:48.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:49:48.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:48.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:49:48.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:48.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:48.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:49:48.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:49:48.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:48.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:48.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:49:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:49:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:49:49.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:49.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:49:49.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:49.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:49.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:49:49.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:49:49.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:49.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:49:49.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:49.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:49.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:49:49.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:49:49.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 239 2024-08-25T19:49:49.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 240 2024-08-25T19:49:49.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:49.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:50.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 238 2024-08-25T19:49:50.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:50.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:49:50.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:50.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:50.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:49:50.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:49:50.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:50.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:50.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:49:50.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:50.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:50.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:49:50.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:49:50.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 242 2024-08-25T19:49:50.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:50.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 241 2024-08-25T19:49:50.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:51.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 245 2024-08-25T19:49:51.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 244 2024-08-25T19:49:51.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:51.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:49:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:49:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:49:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:49:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:49:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:49:51.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 243 2024-08-25T19:49:51.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:51.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 246 2024-08-25T19:49:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 247 2024-08-25T19:49:52.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:52.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:52.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:52.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:49:52.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:52.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:52.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:49:52.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:49:52.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:52.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:49:52.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:52.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:52.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:49:52.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:49:53.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:53.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:49:53.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:53.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:53.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:49:53.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:49:53.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:53.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:49:53.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:53.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:53.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:49:53.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:49:53.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:53.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:49:53.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:53.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:53.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:49:53.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:49:54.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:54.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:49:54.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:54.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:54.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:49:54.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:49:54.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:54.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:49:54.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:54.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:54.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:49:54.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:49:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 250 2024-08-25T19:49:54.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:54.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:54.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:49:54.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:54.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:54.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:49:54.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:49:55.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:55.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:49:55.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:55.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:55.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:49:55.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:49:55.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:55.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:49:55.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:55.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:55.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:49:55.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:49:55.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 248 2024-08-25T19:49:55.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 249 2024-08-25T19:49:55.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:55.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:55.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 252 2024-08-25T19:49:55.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:55.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 251 2024-08-25T19:49:56.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:56.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 255 2024-08-25T19:49:56.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:56.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 254 2024-08-25T19:49:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 253 2024-08-25T19:49:57.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:57.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:57.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:57.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:49:57.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:57.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:57.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:49:57.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:49:57.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 256 2024-08-25T19:49:57.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 257 2024-08-25T19:49:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:49:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:49:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:49:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:49:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:49:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:49:58.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:58.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:49:58.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:58.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:58.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:49:58.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:49:59.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:59.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:49:59.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:59.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:59.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:49:59.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:49:59.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 259 2024-08-25T19:49:59.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 260 2024-08-25T19:49:59.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:59.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:49:59.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:49:59.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:49:59.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:49:59.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:49:59.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:49:59.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:49:59.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 258 2024-08-25T19:50:00.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:00.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:50:00.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:00.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:00.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:50:00.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:50:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 262 2024-08-25T19:50:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:50:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:50:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:50:00.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:00.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 261 2024-08-25T19:50:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:50:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:50:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:50:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:00.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:50:00.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:00.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:50:00.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:50:01.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:01.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 265 2024-08-25T19:50:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 264 2024-08-25T19:50:01.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:01.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:01.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 263 2024-08-25T19:50:01.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:02.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 267 2024-08-25T19:50:02.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 266 2024-08-25T19:50:02.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:50:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:50:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:50:02.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:02.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:02.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:50:02.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:02.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:02.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:50:02.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:50:03.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:03.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:50:03.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:03.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:03.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:50:03.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:50:03.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:03.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:50:03.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:03.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:03.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:50:03.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:50:04.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:04.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:50:04.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:04.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:04.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:50:04.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:50:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:50:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:50:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:50:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:50:04.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:04.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:04.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:50:04.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:50:04.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 270 2024-08-25T19:50:04.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:04.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 269 2024-08-25T19:50:04.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 268 2024-08-25T19:50:05.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:05.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:50:05.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:05.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:05.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:50:05.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:50:05.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:05.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:05.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:50:05.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:05.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:05.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:50:05.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:50:05.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:50:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:50:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:50:05.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 272 2024-08-25T19:50:05.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 271 2024-08-25T19:50:05.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:06.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 275 2024-08-25T19:50:06.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 273 2024-08-25T19:50:07.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 274 2024-08-25T19:50:07.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:07.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:07.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:07.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:50:07.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:07.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:07.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:50:07.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:50:07.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 276 2024-08-25T19:50:08.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 277 2024-08-25T19:50:08.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:08.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:50:08.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:08.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:08.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:50:08.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:50:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:50:08.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:08.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:08.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:50:08.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:50:08.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:08.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:08.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:50:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:50:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:50:09.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:09.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:50:09.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:09.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:09.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:50:09.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:50:09.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:09.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:50:09.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:09.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:09.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:50:09.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:50:09.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 280 2024-08-25T19:50:09.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 279 2024-08-25T19:50:09.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:09.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:10.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 278 2024-08-25T19:50:10.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:10.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:50:10.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:10.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:10.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:50:10.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:50:10.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:10.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:10.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:50:10.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:10.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:10.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:50:10.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:50:10.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 282 2024-08-25T19:50:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:10.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 281 2024-08-25T19:50:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:11.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 284 2024-08-25T19:50:11.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 285 2024-08-25T19:50:11.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:11.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:50:11.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:11.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:11.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:50:11.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:50:11.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:11.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:50:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:50:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:50:12.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 283 2024-08-25T19:50:12.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:12.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 287 2024-08-25T19:50:12.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 286 2024-08-25T19:50:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:50:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:50:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:50:12.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:12.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:12.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:50:12.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:50:12.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:50:12.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:12.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:13.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:13.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:50:13.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:13.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:13.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:50:13.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:50:13.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:13.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:50:13.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:13.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:13.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:50:13.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:50:14.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:14.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:50:14.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:14.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:14.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:50:14.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:50:14.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:14.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:50:14.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:14.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:14.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:50:14.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:50:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:50:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:50:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:50:14.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 290 2024-08-25T19:50:14.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:15.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 289 2024-08-25T19:50:15.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 288 2024-08-25T19:50:15.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:15.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:50:15.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:15.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:15.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:50:15.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:50:15.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:15.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:50:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:50:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:50:15.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:15.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:50:15.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:15.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:15.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:50:15.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:50:15.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 292 2024-08-25T19:50:15.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:15.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 291 2024-08-25T19:50:16.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:16.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 295 2024-08-25T19:50:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:16.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 294 2024-08-25T19:50:16.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 293 2024-08-25T19:50:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:17.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:17.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:17.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:50:17.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:17.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:17.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:50:17.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:50:17.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 297 2024-08-25T19:50:17.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 296 2024-08-25T19:50:18.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:18.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:18.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:18.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:50:18.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:18.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:18.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:50:18.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:50:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:50:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:50:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:50:18.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:18.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:50:18.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:18.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:18.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:50:18.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:50:19.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 300 2024-08-25T19:50:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:50:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:50:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:50:19.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 299 2024-08-25T19:50:19.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:19.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:19.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:50:19.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:19.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:19.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:50:19.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:50:19.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 298 2024-08-25T19:50:20.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:20.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:50:20.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:20.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:20.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:50:20.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:50:20.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:20.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:50:20.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:20.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:20.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:50:20.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:50:20.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:20.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 302 2024-08-25T19:50:20.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:20.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 301 2024-08-25T19:50:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:50:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:50:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:50:21.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:21.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:50:21.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 23357 2024-08-25T19:50:21.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 304 2024-08-25T19:50:21.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 305 2024-08-25T19:50:21.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:21.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 303 2024-08-25T19:50:22.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:22.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 306 2024-08-25T19:50:22.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:22.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:50:22.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:22.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:22.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:50:22.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:50:22.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:22.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:50:22.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:22.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:22.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:50:22.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:50:22.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:23.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:50:23.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 23621 2024-08-25T19:50:23.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:23.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:50:23.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:23.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:23.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:50:23.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:50:24.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:50:24.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 23773 2024-08-25T19:50:24.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:24.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:50:24.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:24.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:24.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:50:24.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:50:24.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:24.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:50:24.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:24.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:24.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:50:24.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:50:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 308 2024-08-25T19:50:25.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:50:25.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 23914 2024-08-25T19:50:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 307 2024-08-25T19:50:25.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:25.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:50:25.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:50:25.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:50:25.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:50:25.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:50:25.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:50:25.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:25.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 309 2024-08-25T19:50:26.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:27.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 310 2024-08-25T19:50:27.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 311 2024-08-25T19:50:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:27.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:28.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 312 2024-08-25T19:50:28.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:50:28.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:50:28.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 24066 2024-08-25T19:50:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:50:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 24219 2024-08-25T19:50:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:50:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 24392 2024-08-25T19:50:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:50:30.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 24546 2024-08-25T19:50:30.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:50:30.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 24719 2024-08-25T19:50:31.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T19:50:31.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T19:50:31.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 23226 2024-08-25T19:52:56.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T19:52:56.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T19:52:56.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T19:52:56.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 23358 2024-08-25T19:52:57.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T19:52:57.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T19:52:57.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T19:52:57.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 23622 2024-08-25T19:52:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T19:52:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T19:52:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T19:52:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 23774 2024-08-25T19:52:59.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T19:52:59.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T19:52:59.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T19:52:59.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 23915 2024-08-25T19:53:00.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T19:53:00.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T19:53:00.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T19:53:00.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 24067 2024-08-25T19:53:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T19:53:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T19:53:01.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T19:53:01.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 24220 2024-08-25T19:53:02.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T19:53:02.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T19:53:02.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T19:53:02.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 24393 2024-08-25T19:53:03.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T19:53:03.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T19:53:03.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T19:53:03.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 24547 2024-08-25T19:53:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T19:53:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T19:53:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T19:53:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 24720 2024-08-25T19:53:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T19:53:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T19:53:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T19:53:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:53:05.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T19:53:05.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:53:05.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 1 2024-08-25T19:53:05.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd0 2024-08-25T19:53:05.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary1 2024-08-25T19:53:05.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary1 2024-08-25T19:53:05.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T19:53:05.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T19:53:05.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary1 2024-08-25T19:53:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:53:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 2 2024-08-25T19:53:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd1 2024-08-25T19:53:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary2 2024-08-25T19:53:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary2 2024-08-25T19:53:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T19:53:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary2 2024-08-25T19:53:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T19:53:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:53:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 3 2024-08-25T19:53:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd2 2024-08-25T19:53:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary3 2024-08-25T19:53:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary3 2024-08-25T19:53:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T19:53:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary3 2024-08-25T19:53:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T19:53:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 4 2024-08-25T19:53:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:53:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd3 2024-08-25T19:53:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary4 2024-08-25T19:53:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary4 2024-08-25T19:53:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T19:53:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary4 2024-08-25T19:53:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T19:53:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:53:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 5 2024-08-25T19:53:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd4 2024-08-25T19:53:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary5 2024-08-25T19:53:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary5 2024-08-25T19:53:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T19:53:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary5 2024-08-25T19:53:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T19:53:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:53:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 6 2024-08-25T19:53:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd5 2024-08-25T19:53:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary6 2024-08-25T19:53:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary6 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary6 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 7 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd6 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary7 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary7 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary7 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:53:05.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 8 2024-08-25T19:53:05.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd7 2024-08-25T19:53:05.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T19:53:05.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary8 2024-08-25T19:53:05.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary8 2024-08-25T19:53:05.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T19:53:05.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary8 2024-08-25T19:53:05.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 9 2024-08-25T19:53:05.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:53:05.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T19:53:05.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd8 2024-08-25T19:53:05.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary9 2024-08-25T19:53:05.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary9 2024-08-25T19:53:05.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T19:53:05.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary9 2024-08-25T19:53:05.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 10 2024-08-25T19:53:05.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T19:53:05.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 30687 2024-08-25T19:53:05.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd9 2024-08-25T19:53:05.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary10 2024-08-25T19:53:05.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary10 2024-08-25T19:53:05.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T19:53:05.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary10 2024-08-25T19:53:06.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T19:53:06.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:53:06.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T19:53:06.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:53:06.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T19:53:06.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:53:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T19:53:06.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:53:06.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:53:06.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T19:53:06.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:53:06.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T19:53:06.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T19:53:06.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:53:06.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T19:53:06.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:53:06.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T19:53:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:53:06.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:53:06.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T19:54:02.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=05b3f2820aef556cc9cdbc64661f9b79 2024-08-25T19:54:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-08-25T19:54:02.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary4 2024-08-25T19:54:02.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:02.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:02.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:02.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-08-25T19:54:03.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T19:54:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-08-25T19:54:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T19:54:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:04.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:04.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T19:54:04.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T19:54:04.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b75218c8-fbe6-4268-8a2c-89c6f9d197fa 2024-08-25T19:54:04.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:04.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:04.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:04.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:04.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:04.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:04.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:04.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T19:54:05.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=ad837d930c1516a35c05b1e1d70ac064 2024-08-25T19:54:05.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-08-25T19:54:06.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary5 2024-08-25T19:54:06.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:06.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:06.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:54:06.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-08-25T19:54:06.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T19:54:06.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:06.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:06.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:06.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T19:54:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T19:54:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b75218c8-fbe6-4268-8a2c-89c6f9d197fa 2024-08-25T19:54:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:06.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:06.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:06.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T19:54:07.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T19:54:07.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=29373fd85709709aac8876a4c47b30f5 2024-08-25T19:54:07.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-08-25T19:54:07.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-08-25T19:54:07.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:07.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:07.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:54:07.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:07.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:07.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:07.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:07.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:07.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary3 2024-08-25T19:54:07.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:07.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:07.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:54:07.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-08-25T19:54:07.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=b52bef1c7d94c1855742bd61cd3b5170 2024-08-25T19:54:07.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-08-25T19:54:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary6 2024-08-25T19:54:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:54:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-08-25T19:54:08.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T19:54:08.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-08-25T19:54:08.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:08.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:08.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:54:08.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:08.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:08.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:08.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:08.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T19:54:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:54:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T19:54:08.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=0f91d3d3a5646478d7d73510a5872ddc 2024-08-25T19:54:08.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-08-25T19:54:08.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=1014801a1f973e46079affb8c1a678ae 2024-08-25T19:54:08.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-08-25T19:54:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T19:54:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: af8e54d2-12d3-45f1-8e69-422f660f56dc 2024-08-25T19:54:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:08.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:08.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:08.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T19:54:08.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary8 2024-08-25T19:54:08.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:08.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:08.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:54:08.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-08-25T19:54:08.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary1 2024-08-25T19:54:08.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:08.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:08.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:54:08.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-08-25T19:54:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=1a4e68281e1412188c419f9f40121cbf 2024-08-25T19:54:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-08-25T19:54:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary7 2024-08-25T19:54:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:54:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-08-25T19:54:09.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=c26e7d4f5b668a03d702444717a0ee96 2024-08-25T19:54:09.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-08-25T19:54:09.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T19:54:09.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T19:54:09.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T19:54:09.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T19:54:09.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:54:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:09.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:09.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T19:54:09.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-08-25T19:54:09.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:09.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:09.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:54:09.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:09.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:09.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:09.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:09.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-08-25T19:54:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:54:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:09.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-08-25T19:54:09.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:09.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:09.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:54:09.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:09.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:09.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:09.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:09.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:09.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary9 2024-08-25T19:54:09.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:54:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-08-25T19:54:09.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T19:54:09.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b1eaf95b-5076-4fa7-856b-29a8cc179965 2024-08-25T19:54:09.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:09.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:09.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:09.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:09.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:09.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T19:54:09.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=804922f3a02d8f67d9b4f440368d563a 2024-08-25T19:54:09.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-08-25T19:54:10.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary2 2024-08-25T19:54:10.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:10.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:10.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:54:10.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-08-25T19:54:10.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T19:54:10.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T19:54:10.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-08-25T19:54:10.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:10.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:10.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:54:10.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:10.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:10.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:10.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:10.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:10.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T19:54:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-08-25T19:54:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:54:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:10.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-08-25T19:54:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:54:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:10.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T19:54:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:54:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:10.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T19:54:10.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T19:54:10.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:10.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:10.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:54:10.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:10.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:10.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:10.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:10.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:10.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T19:54:10.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T19:54:10.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:10.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:10.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:54:10.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:10.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:10.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:10.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:10.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:10.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T19:54:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T19:54:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 271f3fac-9544-4101-b3db-355632166a97 2024-08-25T19:54:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T19:54:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5fdd517d-317f-4c25-b8f4-f5395bd911d6 2024-08-25T19:54:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:10.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T19:54:10.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T19:54:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5641da01-7c6f-4b5d-9e77-32496e6199e6 2024-08-25T19:54:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:10.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:10.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T19:54:10.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:10.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:10.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T19:54:10.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=643372c03a28b141e5fb3e01fb0f3c9c 2024-08-25T19:54:10.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-08-25T19:54:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary10 2024-08-25T19:54:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:54:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-08-25T19:54:10.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T19:54:10.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:10.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:10.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:54:10.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:10.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:10.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:10.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:10.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:10.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T19:54:10.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T19:54:10.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: af8e54d2-12d3-45f1-8e69-422f660f56dc 2024-08-25T19:54:10.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:10.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:10.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:10.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:10.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:10.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:10.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:10.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T19:54:10.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T19:54:10.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:10.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:10.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:10.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:10.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:10.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:10.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:10.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:10.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T19:54:10.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T19:54:10.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b75218c8-fbe6-4268-8a2c-89c6f9d197fa 2024-08-25T19:54:10.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:10.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:10.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:10.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:10.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:10.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:10.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:10.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:11.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T19:54:11.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:11.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:11.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:54:11.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:11.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:11.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:11.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:11.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:11.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T19:54:11.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T19:54:11.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T19:54:11.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:11.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:11.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:54:11.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:11.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:11.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:11.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:11.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:11.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T19:54:11.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T19:54:11.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:11.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:11.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:54:11.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:11.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T19:54:11.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-08-25T19:54:11.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:11.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:54:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:11.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T19:54:11.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 73264916-cfd5-40d0-83ae-ff7d92659663 2024-08-25T19:54:11.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:11.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:11.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:11.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:11.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:11.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:11.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:11.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T19:54:11.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T19:54:11.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: bbfc30c2-dc11-4de7-ada3-fc4241ba8134 2024-08-25T19:54:11.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:11.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:11.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:11.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:11.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:11.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:11.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:11.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T19:54:11.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T19:54:11.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 540d8938-7a8c-477e-8fc3-e65c72e78c93 2024-08-25T19:54:11.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:11.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:11.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:11.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:11.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:11.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:11.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:11.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T19:54:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T19:54:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:54:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T19:54:11.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T19:54:11.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b1eaf95b-5076-4fa7-856b-29a8cc179965 2024-08-25T19:54:11.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:11.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:11.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:11.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:11.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:11.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:11.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:11.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T19:54:12.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T19:54:12.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:12.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:12.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:54:12.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:12.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:12.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:12.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:12.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:12.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T19:54:12.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T19:54:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: fad93bc2-e996-49cb-a574-393a1cbbe56a 2024-08-25T19:54:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:12.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:12.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:12.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T19:54:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T19:54:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:54:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:12.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:12.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T19:54:12.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T19:54:12.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:12.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:12.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:54:12.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:12.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:12.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:12.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:12.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:12.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T19:54:12.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T19:54:12.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:12.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:12.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:54:12.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:12.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:12.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:12.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:12.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:12.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T19:54:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T19:54:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 271f3fac-9544-4101-b3db-355632166a97 2024-08-25T19:54:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:12.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:12.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:12.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T19:54:12.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T19:54:12.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5641da01-7c6f-4b5d-9e77-32496e6199e6 2024-08-25T19:54:12.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:12.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:12.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:12.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:12.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:12.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T19:54:12.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5fdd517d-317f-4c25-b8f4-f5395bd911d6 2024-08-25T19:54:12.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:12.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:12.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:12.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:12.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:12.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:12.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:12.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T19:54:12.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:12.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:12.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T19:54:13.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T19:54:13.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:13.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:13.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:54:13.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:13.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:13.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:13.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:13.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:13.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T19:54:13.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T19:54:13.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:13.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:13.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:54:13.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:13.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:13.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:13.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:13.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:13.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T19:54:13.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T19:54:13.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:13.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:13.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:54:13.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:13.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:13.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:13.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:13.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:13.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T19:54:13.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T19:54:13.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 73264916-cfd5-40d0-83ae-ff7d92659663 2024-08-25T19:54:13.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:13.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:13.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:13.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:13.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:13.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:13.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:13.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T19:54:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T19:54:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: bbfc30c2-dc11-4de7-ada3-fc4241ba8134 2024-08-25T19:54:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T19:54:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 540d8938-7a8c-477e-8fc3-e65c72e78c93 2024-08-25T19:54:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T19:54:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T19:54:14.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T19:54:14.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:14.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:14.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:54:14.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:14.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:14.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:14.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:14.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:14.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T19:54:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T19:54:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: fad93bc2-e996-49cb-a574-393a1cbbe56a 2024-08-25T19:54:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:14.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:14.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:14.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T19:54:14.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T19:54:14.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:14.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:14.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:54:14.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:14.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:14.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:14.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:14.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:14.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T19:54:14.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T19:54:14.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: af8e54d2-12d3-45f1-8e69-422f660f56dc 2024-08-25T19:54:14.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:14.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:14.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:14.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:14.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:14.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:14.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:14.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:15.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T19:54:15.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:15.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:15.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:54:15.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:15.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:15.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:15.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:15.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:15.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T19:54:15.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T19:54:15.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b1eaf95b-5076-4fa7-856b-29a8cc179965 2024-08-25T19:54:15.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:15.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:15.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:15.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:15.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:15.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:15.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:15.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:16.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T19:54:16.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:16.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:54:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T19:54:16.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T19:54:16.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:16.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:16.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:16.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T19:54:16.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:16.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T19:54:16.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T19:54:16.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 271f3fac-9544-4101-b3db-355632166a97 2024-08-25T19:54:16.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:16.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:16.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:16.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:16.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:16.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T19:54:16.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5641da01-7c6f-4b5d-9e77-32496e6199e6 2024-08-25T19:54:16.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:16.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:16.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:16.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:16.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:16.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T19:54:16.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5fdd517d-317f-4c25-b8f4-f5395bd911d6 2024-08-25T19:54:16.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:16.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:16.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:16.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:16.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:16.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:16.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:16.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:16.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:16.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:16.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:16.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:16.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:16.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:17.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T19:54:17.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:17.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:17.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:54:17.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:17.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:17.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:17.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:17.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:17.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T19:54:17.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T19:54:17.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:17.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:17.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:54:17.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T19:54:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:54:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:17.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:17.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T19:54:17.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:17.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T19:54:17.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T19:54:17.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: bbfc30c2-dc11-4de7-ada3-fc4241ba8134 2024-08-25T19:54:17.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:17.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:17.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:17.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:17.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:17.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:17.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:17.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:17.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T19:54:17.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 540d8938-7a8c-477e-8fc3-e65c72e78c93 2024-08-25T19:54:17.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:17.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:17.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:17.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:17.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:17.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T19:54:17.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 73264916-cfd5-40d0-83ae-ff7d92659663 2024-08-25T19:54:17.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:17.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:17.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:17.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:17.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:17.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:17.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:17.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:17.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:17.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:17.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:18.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T19:54:18.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:18.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:18.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:54:18.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:18.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:18.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:18.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:18.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:18.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T19:54:18.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T19:54:18.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: fad93bc2-e996-49cb-a574-393a1cbbe56a 2024-08-25T19:54:18.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:18.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:18.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:18.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:18.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:18.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:18.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:18.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:18.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T19:54:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T19:54:19.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T19:54:19.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b75218c8-fbe6-4268-8a2c-89c6f9d197fa 2024-08-25T19:54:19.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:19.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:19.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:19.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:19.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:19.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:19.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:19.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:22.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T19:54:22.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:22.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:54:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T19:54:22.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T19:54:22.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: af8e54d2-12d3-45f1-8e69-422f660f56dc 2024-08-25T19:54:22.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:22.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:22.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:22.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:22.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:22.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:22.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:22.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:23.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T19:54:23.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:23.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:23.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:54:23.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:23.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:23.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:23.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T19:54:23.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T19:54:23.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b1eaf95b-5076-4fa7-856b-29a8cc179965 2024-08-25T19:54:23.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:23.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:23.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:23.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:23.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:23.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:23.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:23.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:24.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T19:54:24.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:24.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:24.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:54:24.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:24.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:24.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T19:54:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:54:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T19:54:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:24.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:24.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T19:54:24.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:24.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:54:24.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:24.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:24.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:24.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:24.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:24.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T19:54:24.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:24.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T19:54:24.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T19:54:24.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 271f3fac-9544-4101-b3db-355632166a97 2024-08-25T19:54:24.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:24.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:24.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:24.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:24.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:24.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T19:54:24.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5fdd517d-317f-4c25-b8f4-f5395bd911d6 2024-08-25T19:54:24.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:24.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:24.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:24.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:24.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:24.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T19:54:24.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5641da01-7c6f-4b5d-9e77-32496e6199e6 2024-08-25T19:54:24.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:24.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:24.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:24.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:24.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:24.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:24.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:24.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:25.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T19:54:25.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:25.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:25.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:54:25.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:25.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:25.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:25.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:25.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:25.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T19:54:25.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T19:54:25.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:25.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:25.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:54:25.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:25.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T19:54:25.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T19:54:25.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: bbfc30c2-dc11-4de7-ada3-fc4241ba8134 2024-08-25T19:54:25.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:25.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:25.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:25.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:25.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:25.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T19:54:25.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 540d8938-7a8c-477e-8fc3-e65c72e78c93 2024-08-25T19:54:25.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:25.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:25.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 73264916-cfd5-40d0-83ae-ff7d92659663 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:25.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:25.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:25.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:25.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T19:54:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:54:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:26.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:26.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:26.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:26.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:26.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T19:54:26.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T19:54:26.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: fad93bc2-e996-49cb-a574-393a1cbbe56a 2024-08-25T19:54:26.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:26.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:26.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:26.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:26.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:26.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:26.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:26.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:27.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T19:54:27.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:27.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:27.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:27.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:27.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:27.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:27.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:27.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:27.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T19:54:27.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T19:54:27.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b75218c8-fbe6-4268-8a2c-89c6f9d197fa 2024-08-25T19:54:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T19:54:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T19:54:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:53:58' 2024-08-25T19:54:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:27.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T19:54:27.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:27.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T19:54:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T19:54:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:54:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:30.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:30.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:30.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T19:54:30.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T19:54:30.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: af8e54d2-12d3-45f1-8e69-422f660f56dc 2024-08-25T19:54:30.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:30.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:30.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:30.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:30.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:30.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:30.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:30.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:30.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:30.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:30.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:30.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:30.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:30.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:30.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:30.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:30.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:30.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:30.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:30.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:54:30.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:30.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:30.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:30.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:30.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:31.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:31.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:31.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T19:54:31.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:31.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T19:54:31.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:31.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:31.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:31.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:31.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-08-25T19:54:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:54:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:31.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:31.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:31.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T19:54:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:54:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T19:54:31.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T19:54:31.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b1eaf95b-5076-4fa7-856b-29a8cc179965 2024-08-25T19:54:31.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:31.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:31.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:31.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:31.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:31.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:31.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:31.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:31.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:31.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:31.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:31.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:54:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:31.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:31.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:31.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:31.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:31.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T19:54:31.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:31.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T19:54:31.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:31.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:31.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:31.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:31.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:31.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:31.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-08-25T19:54:31.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:31.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:31.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:54:31.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:31.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:31.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:31.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:31.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:32.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-08-25T19:54:32.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:32.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:54:32.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:32.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:32.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:32.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:32.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-08-25T19:54:32.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T19:54:32.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: af8e54d2-12d3-45f1-8e69-422f660f56dc 2024-08-25T19:54:32.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:32.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:32.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:32.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:32.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:32.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:32.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:32.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:32.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:32.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:32.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:32.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:32.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:32.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:32.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:32.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:32.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:32.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:54:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:32.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:32.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:32.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T19:54:32.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:32.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T19:54:32.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:32.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:32.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:32.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:32.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:32.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:32.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary5 2024-08-25T19:54:32.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:32.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:54:32.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T19:54:32.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-08-25T19:54:32.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T19:54:32.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:32.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:54:32.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:32.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:32.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:32.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:32.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T19:54:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T19:54:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:54:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T19:54:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:54:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T19:54:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T19:54:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T19:54:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5641da01-7c6f-4b5d-9e77-32496e6199e6 2024-08-25T19:54:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T19:54:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 271f3fac-9544-4101-b3db-355632166a97 2024-08-25T19:54:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T19:54:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5fdd517d-317f-4c25-b8f4-f5395bd911d6 2024-08-25T19:54:32.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:32.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:32.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:32.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:32.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:32.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:32.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:32.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:32.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:32.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:32.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:32.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:32.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:32.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:32.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:32.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:32.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:32.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:32.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:32.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:32.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:32.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:32.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:32.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:32.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:32.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:32.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:32.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:32.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:32.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:32.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:32.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:54:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:32.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:32.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:32.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:32.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:32.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:54:32.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:32.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:32.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:32.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T19:54:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:54:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:54:32.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-08-25T19:54:32.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:32.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:32.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T19:54:32.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:32.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-08-25T19:54:32.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T19:54:32.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:32.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:54:32.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:32.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:32.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:32.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:32.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-08-25T19:54:32.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:32.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:32.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:32.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:32.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:32.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:32.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-08-25T19:54:32.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:32.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:54:32.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:32.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:32.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:32.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:32.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T19:54:32.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:32.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T19:54:32.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:32.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:32.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T19:54:32.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:32.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:32.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:32.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:32.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T19:54:32.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:32.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:32.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:32.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-08-25T19:54:32.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:32.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:54:32.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:32.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:32.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:32.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:32.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:32.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:32.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:32.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-08-25T19:54:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:54:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:32.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd0 2024-08-25T19:54:32.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T19:54:32.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:54:32.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T19:54:32.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b1eaf95b-5076-4fa7-856b-29a8cc179965 2024-08-25T19:54:32.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:32.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:32.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:32.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:32.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:32.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:32.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:32.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:32.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:32.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:32.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:32.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:32.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:32.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:32.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:32.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:32.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:32.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:32.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:32.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:32.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:54:32.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:32.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:32.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:32.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:32.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:32.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:32.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:32.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T19:54:32.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:33.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T19:54:33.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:33.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:33.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:33.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:33.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:33.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:33.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary3 2024-08-25T19:54:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:54:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T19:54:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-08-25T19:54:33.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T19:54:33.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:33.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:54:33.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:33.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:33.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:33.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:33.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T19:54:33.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T19:54:33.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:33.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:54:33.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:33.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:33.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:33.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:33.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T19:54:33.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T19:54:33.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:33.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:54:33.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:33.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:33.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:33.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:33.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T19:54:33.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T19:54:33.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 540d8938-7a8c-477e-8fc3-e65c72e78c93 2024-08-25T19:54:33.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:33.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:33.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:33.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:33.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:33.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:33.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T19:54:33.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: bbfc30c2-dc11-4de7-ada3-fc4241ba8134 2024-08-25T19:54:33.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:33.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:33.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:33.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:33.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:33.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:33.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:33.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:33.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T19:54:33.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 73264916-cfd5-40d0-83ae-ff7d92659663 2024-08-25T19:54:33.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:33.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:33.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:33.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:33.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:33.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:33.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:33.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:33.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:33.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:33.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:33.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:33.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:33.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:33.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:33.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:33.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:54:33.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:33.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:33.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:33.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:33.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:33.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:33.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:54:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:33.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:33.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:33.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:33.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:33.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:33.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:54:33.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T19:54:33.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:54:33.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:54:33.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-08-25T19:54:33.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-08-25T19:54:33.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:54:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-08-25T19:54:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T19:54:33.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:33.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:33.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:33.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T19:54:33.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T19:54:33.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:33.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:33.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:33.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:33.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T19:54:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:33.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T19:54:33.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:33.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-08-25T19:54:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:54:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:33.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:33.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:33.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:33.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:33.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T19:54:33.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-08-25T19:54:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:54:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-08-25T19:54:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:54:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-08-25T19:54:33.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:33.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:33.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:33.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:33.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:33.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:33.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-08-25T19:54:33.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:33.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:33.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:33.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:33.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-08-25T19:54:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd1 2024-08-25T19:54:33.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T19:54:33.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:54:33.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T19:54:33.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 271f3fac-9544-4101-b3db-355632166a97 2024-08-25T19:54:33.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:33.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:33.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:33.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:33.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:33.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:33.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:33.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:33.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:33.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:33.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:33.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:33.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:33.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:33.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:33.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:33.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:54:33.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:33.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:33.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:33.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:33.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T19:54:33.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5fdd517d-317f-4c25-b8f4-f5395bd911d6 2024-08-25T19:54:33.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:33.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:33.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:33.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:33.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:33.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:33.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:33.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:33.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:33.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:33.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:33.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:33.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:33.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T19:54:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5641da01-7c6f-4b5d-9e77-32496e6199e6 2024-08-25T19:54:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:33.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:33.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:33.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:33.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:33.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:33.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:33.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:54:33.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:33.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:33.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:33.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:33.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:33.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:54:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:33.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:34.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:34.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T19:54:34.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:34.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T19:54:34.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:34.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:34.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:34.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:34.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:34.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary1 2024-08-25T19:54:34.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:34.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:34.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:54:34.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T19:54:34.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-08-25T19:54:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T19:54:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T19:54:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T19:54:34.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:34.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T19:54:34.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:34.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:34.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:34.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:34.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary8 2024-08-25T19:54:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:54:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T19:54:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-08-25T19:54:34.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:34.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:34.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:34.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary6 2024-08-25T19:54:34.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:34.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:34.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:54:34.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T19:54:34.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-08-25T19:54:34.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T19:54:34.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:34.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:34.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:54:34.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:34.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:34.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:34.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:34.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:34.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T19:54:34.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T19:54:34.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: fad93bc2-e996-49cb-a574-393a1cbbe56a 2024-08-25T19:54:34.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:34.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:34.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:34.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:34.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:34.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:34.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:34.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:34.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:54:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:34.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:34.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:34.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T19:54:34.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T19:54:34.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T19:54:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:54:34.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:54:34.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:54:34.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:54:34.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-08-25T19:54:34.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-08-25T19:54:34.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:54:34.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:54:34.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-08-25T19:54:34.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:34.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:34.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T19:54:34.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T19:54:34.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:34.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:34.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:34.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:34.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:34.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:34.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-08-25T19:54:34.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:34.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:34.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:54:34.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:34.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:34.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:34.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:34.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:34.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-08-25T19:54:34.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:34.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:34.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:54:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:34.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:34.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-08-25T19:54:34.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-08-25T19:54:34.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:34.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:34.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:54:34.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:34.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:34.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:34.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:34.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:34.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-08-25T19:54:34.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-08-25T19:54:34.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:34.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:34.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:54:34.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:34.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:34.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:34.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:34.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:34.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-08-25T19:54:34.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd2 2024-08-25T19:54:34.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd3 2024-08-25T19:54:34.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T19:54:34.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:54:34.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T19:54:34.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:54:35.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd4 2024-08-25T19:54:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T19:54:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:54:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T19:54:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 540d8938-7a8c-477e-8fc3-e65c72e78c93 2024-08-25T19:54:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T19:54:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 73264916-cfd5-40d0-83ae-ff7d92659663 2024-08-25T19:54:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T19:54:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: bbfc30c2-dc11-4de7-ada3-fc4241ba8134 2024-08-25T19:54:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:35.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:35.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:35.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:35.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:35.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:35.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:35.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:35.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:35.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:35.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:35.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:35.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:35.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:35.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:35.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:35.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:35.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:35.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:35.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:35.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:35.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:35.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:35.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:35.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:54:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:35.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:54:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:35.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:35.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:35.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:35.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:35.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:35.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:35.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:35.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:54:35.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:35.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:35.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:35.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:35.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T19:54:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T19:54:35.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T19:54:35.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b75218c8-fbe6-4268-8a2c-89c6f9d197fa 2024-08-25T19:54:35.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:35.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:35.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:35.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:35.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:35.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:35.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:35.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:35.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T19:54:35.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:35.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:35.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:35.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T19:54:35.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:35.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T19:54:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:35.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:35.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T19:54:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:35.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:35.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:35.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T19:54:35.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:35.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:35.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:35.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:35.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:35.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:35.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:35.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T19:54:35.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:35.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:35.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:35.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:35.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary2 2024-08-25T19:54:35.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:35.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:35.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:54:35.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T19:54:35.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-08-25T19:54:35.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:35.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:35.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:54:35.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:35.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:35.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:35.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:35.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:35.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:35.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:35.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:35.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:35.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary7 2024-08-25T19:54:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:54:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T19:54:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-08-25T19:54:35.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:35.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:35.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:35.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:35.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:35.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary9 2024-08-25T19:54:35.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:35.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:35.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:54:35.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T19:54:35.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-08-25T19:54:35.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:35.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:35.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T19:54:35.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:35.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T19:54:35.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:35.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:35.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:35.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-08-25T19:54:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:54:35.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:54:35.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T19:54:35.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T19:54:35.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T19:54:35.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:54:35.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:54:35.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-08-25T19:54:35.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:54:35.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:54:35.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-08-25T19:54:35.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:54:35.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:54:35.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-08-25T19:54:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-08-25T19:54:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:54:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-08-25T19:54:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T19:54:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: fad93bc2-e996-49cb-a574-393a1cbbe56a 2024-08-25T19:54:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:36.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:36.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:36.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:36.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:36.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:36.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd5 2024-08-25T19:54:36.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T19:54:36.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd6 2024-08-25T19:54:36.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:36.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:54:36.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T19:54:36.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:36.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:36.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:54:36.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:36.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:36.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:36.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:36.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:36.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:54:36.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:36.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:36.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:36.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:36.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:36.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd7 2024-08-25T19:54:36.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:54:36.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T19:54:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:36.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T19:54:36.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:36.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T19:54:36.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:36.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:36.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:36.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:36.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:36.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:36.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary10 2024-08-25T19:54:36.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:36.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:36.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:54:36.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T19:54:36.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-08-25T19:54:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-08-25T19:54:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:54:36.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:54:36.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:36.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:36.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-08-25T19:54:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T19:54:36.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:54:36.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:54:36.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-08-25T19:54:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T19:54:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b75218c8-fbe6-4268-8a2c-89c6f9d197fa 2024-08-25T19:54:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T19:54:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T19:54:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:54:36.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:54:28' 2024-08-25T19:54:36.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T19:54:36.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T19:54:36.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:54:36.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T19:54:36.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:54:36.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:36.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:54:36.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:36.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:54:36.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:54:36.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:54:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:54:36.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:36.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-08-25 19:54:28' 2024-08-25T19:54:36.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:36.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:36.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:36.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T19:54:36.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:54:28' 2024-08-25T19:54:36.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:54:36.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:54:36.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T19:54:37.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd8 2024-08-25T19:54:37.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T19:54:37.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:54:37.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 19:54:28' 2024-08-25T19:54:37.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:54:37.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T19:54:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:54:28' 2024-08-25T19:54:37.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T19:54:37.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T19:54:37.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T19:54:37.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T19:54:37.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T19:54:37.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:37.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:54:37.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary4 2024-08-25T19:54:37.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:54:37.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:54:37.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:54:37.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T19:54:37.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-08-25T19:54:37.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T19:54:37.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:54:37.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:54:37.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-08-25T19:54:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd9 2024-08-25T19:54:38.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T19:54:38.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T19:55:28.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=ad837d930c1516a35c05b1e1d70ac064 2024-08-25T19:55:28.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-08-25T19:55:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=29373fd85709709aac8876a4c47b30f5 2024-08-25T19:55:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-08-25T19:55:29.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ ad837d930c1516a35c05b1e1d70ac064 != \a\d\8\3\7\d\9\3\0\c\1\5\1\6\a\3\5\c\0\5\b\1\e\1\d\7\0\a\c\0\6\4 ]] 2024-08-25T19:55:29.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 29373fd85709709aac8876a4c47b30f5 != \2\9\3\7\3\f\d\8\5\7\0\9\7\0\9\a\a\c\8\8\7\6\a\4\c\4\7\b\3\0\f\5 ]] 2024-08-25T19:55:30.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=1014801a1f973e46079affb8c1a678ae 2024-08-25T19:55:30.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-08-25T19:55:30.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 1014801a1f973e46079affb8c1a678ae != \1\0\1\4\8\0\1\a\1\f\9\7\3\e\4\6\0\7\9\a\f\f\b\8\c\1\a\6\7\8\a\e ]] 2024-08-25T19:55:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T19:55:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 30688 2024-08-25T19:55:31.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=b52bef1c7d94c1855742bd61cd3b5170 2024-08-25T19:55:31.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-08-25T19:55:31.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ b52bef1c7d94c1855742bd61cd3b5170 != \b\5\2\b\e\f\1\c\7\d\9\4\c\1\8\5\5\7\4\2\b\d\6\1\c\d\3\b\5\1\7\0 ]] 2024-08-25T19:55:31.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=0f91d3d3a5646478d7d73510a5872ddc 2024-08-25T19:55:31.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-08-25T19:55:31.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=804922f3a02d8f67d9b4f440368d563a 2024-08-25T19:55:31.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-08-25T19:55:31.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 0f91d3d3a5646478d7d73510a5872ddc != \0\f\9\1\d\3\d\3\a\5\6\4\6\4\7\8\d\7\d\7\3\5\1\0\a\5\8\7\2\d\d\c ]] 2024-08-25T19:55:32.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 804922f3a02d8f67d9b4f440368d563a != \8\0\4\9\2\2\f\3\a\0\2\d\8\f\6\7\d\9\b\4\f\4\4\0\3\6\8\d\5\6\3\a ]] 2024-08-25T19:55:32.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T19:55:32.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 30690 2024-08-25T19:55:32.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T19:55:32.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 30692 2024-08-25T19:55:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=1a4e68281e1412188c419f9f40121cbf 2024-08-25T19:55:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-08-25T19:55:32.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 1a4e68281e1412188c419f9f40121cbf != \1\a\4\e\6\8\2\8\1\e\1\4\1\2\1\8\8\c\4\1\9\f\9\f\4\0\1\2\1\c\b\f ]] 2024-08-25T19:55:32.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=c26e7d4f5b668a03d702444717a0ee96 2024-08-25T19:55:32.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-08-25T19:55:33.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ c26e7d4f5b668a03d702444717a0ee96 != \c\2\6\e\7\d\4\f\5\b\6\6\8\a\0\3\d\7\0\2\4\4\4\7\1\7\a\0\e\e\9\6 ]] 2024-08-25T19:55:33.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=643372c03a28b141e5fb3e01fb0f3c9c 2024-08-25T19:55:33.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-08-25T19:55:33.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 643372c03a28b141e5fb3e01fb0f3c9c != \6\4\3\3\7\2\c\0\3\a\2\8\b\1\4\1\e\5\f\b\3\e\0\1\f\b\0\f\3\c\9\c ]] 2024-08-25T19:55:34.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=05b3f2820aef556cc9cdbc64661f9b79 2024-08-25T19:55:34.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-08-25T19:55:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 05b3f2820aef556cc9cdbc64661f9b79 != \0\5\b\3\f\2\8\2\0\a\e\f\5\5\6\c\c\9\c\d\b\c\6\4\6\6\1\f\9\b\7\9 ]] 2024-08-25T19:55:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T19:55:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 30694 2024-08-25T19:55:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T19:55:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 30696 2024-08-25T19:55:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T19:55:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 30698 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 30700 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 30702 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 30704 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:34.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:55:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T19:55:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-08-25T19:55:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:55:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:55:35.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-08-25T19:55:35.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:35.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:35.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:55:35.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:35.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:35.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:35.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:35.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:35.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T19:55:35.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T19:55:35.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 271f3fac-9544-4101-b3db-355632166a97 2024-08-25T19:55:35.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T19:55:35.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3429171.2,"bytes_per_snapshot":68583424.0,"last_snapshot_bytes":68583424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"} 2024-08-25T19:55:35.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:55:35.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:55:28' 2024-08-25T19:55:35.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T19:55:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T19:55:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:55:35.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3429171.2,"bytes_per_snapshot":68583424.0,"last_snapshot_bytes":68583424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"} 2024-08-25T19:55:35.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:55:35.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:35.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:55:35.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:35.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:55:35.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:55:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:55:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:55:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-08-25 19:55:28' 2024-08-25T19:55:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:55:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:55:28' 2024-08-25T19:55:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:55:35.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1271f3fac-9544-4101-b3db-355632166a97up+replayingreplaying, {"bytes_per_second":3429171.2,"bytes_per_snapshot":68583424.0,"last_snapshot_bytes":68583424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+replayingreplaying, {"bytes_per_second":3254681.6,"bytes_per_snapshot":65093632.0,"last_snapshot_bytes":65093632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615676,"remote_snapshot_timestamp":1724615676,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+replayingreplaying, {"bytes_per_second":3227648.0,"bytes_per_snapshot":64552960.0,"last_snapshot_bytes":64552960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+replayingreplaying, {"bytes_per_second":3134873.6,"bytes_per_snapshot":62697472.0,"last_snapshot_bytes":62697472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615673,"remote_snapshot_timestamp":1724615673,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+replayingreplaying, {"bytes_per_second":3342540.8,"bytes_per_snapshot":66850816.0,"last_snapshot_bytes":66850816,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615677,"remote_snapshot_timestamp":1724615677,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+replayingreplaying, {"bytes_per_second":3069337.6,"bytes_per_snapshot":61386752.0,"last_snapshot_bytes":61386752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615672,"remote_snapshot_timestamp":1724615672,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+replayingreplaying, {"bytes_per_second":3536896.0,"bytes_per_snapshot":70737920.0,"last_snapshot_bytes":70737920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+replayingreplaying, {"bytes_per_second":3507200.0,"bytes_per_snapshot":70144000.0,"last_snapshot_bytes":70144000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+replayingreplaying, {"bytes_per_second":3414016.0,"bytes_per_snapshot":68280320.0,"last_snapshot_bytes":68280320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+replayingreplaying, {"bytes_per_second":3334553.6,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28' 2024-08-25T19:55:35.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:55:35.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T19:55:35.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:35.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T19:55:35.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T19:55:35.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:35.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T19:55:35.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:35.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:35.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:35.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary1 2024-08-25T19:55:35.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:55:35.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:35.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:55:35.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-08-25T19:55:35.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-08-25T19:55:35.569 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:35.568+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8bda00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T19:55:35.570 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:35.568+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972000 [3/271f3fac-9544-4101-b3db-355632166a97] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T19:55:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T19:55:36.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-08-25T19:55:36.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:36.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:36.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:55:36.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:36.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T19:55:36.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary1 2024-08-25T19:55:36.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:36.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary1 2024-08-25T19:55:36.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:55:36.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T19:55:36.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-08-25T19:55:36.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:36.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:36.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:55:36.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T19:55:36.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-08-25T19:55:36.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:36.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:36.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:55:36.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:36.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:36.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:36.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:36.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:55:37.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-08-25T19:55:37.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:37.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:37.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:55:37.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:37.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:37.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:37.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:37.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:37.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T19:55:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T19:55:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: bbfc30c2-dc11-4de7-ada3-fc4241ba8134 2024-08-25T19:55:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T19:55:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3227648.0,"bytes_per_snapshot":64552960.0,"last_snapshot_bytes":64552960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"} 2024-08-25T19:55:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:55:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:55:28' 2024-08-25T19:55:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T19:55:37.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T19:55:37.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:55:37.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3227648.0,"bytes_per_snapshot":64552960.0,"last_snapshot_bytes":64552960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"} 2024-08-25T19:55:37.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:55:37.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:37.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:55:37.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:37.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:55:38.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:55:38.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:55:38.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:55:38.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:38.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-08-25 19:55:28' 2024-08-25T19:55:38.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:38.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:38.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:55:38.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:38.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:55:28' 2024-08-25T19:55:38.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:38.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:38.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:55:38.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING941904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+replayingreplaying, {"bytes_per_second":3254681.6,"bytes_per_snapshot":65093632.0,"last_snapshot_bytes":65093632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615676,"remote_snapshot_timestamp":1724615676,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary2bbfc30c2-dc11-4de7-ada3-fc4241ba8134up+replayingreplaying, {"bytes_per_second":3227648.0,"bytes_per_snapshot":64552960.0,"last_snapshot_bytes":64552960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+replayingreplaying, {"bytes_per_second":3134873.6,"bytes_per_snapshot":62697472.0,"last_snapshot_bytes":62697472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615673,"remote_snapshot_timestamp":1724615673,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+replayingreplaying, {"bytes_per_second":3342540.8,"bytes_per_snapshot":66850816.0,"last_snapshot_bytes":66850816,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615677,"remote_snapshot_timestamp":1724615677,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+replayingreplaying, {"bytes_per_second":3069337.6,"bytes_per_snapshot":61386752.0,"last_snapshot_bytes":61386752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615672,"remote_snapshot_timestamp":1724615672,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+replayingreplaying, {"bytes_per_second":3536896.0,"bytes_per_snapshot":70737920.0,"last_snapshot_bytes":70737920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+replayingreplaying, {"bytes_per_second":3507200.0,"bytes_per_snapshot":70144000.0,"last_snapshot_bytes":70144000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+replayingreplaying, {"bytes_per_second":3414016.0,"bytes_per_snapshot":68280320.0,"last_snapshot_bytes":68280320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+replayingreplaying, {"bytes_per_second":3334553.6,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28' 2024-08-25T19:55:38.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:55:38.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T19:55:38.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:38.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T19:55:38.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T19:55:38.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:38.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T19:55:38.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:38.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:38.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:38.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary2 2024-08-25T19:55:38.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:55:38.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:38.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:55:38.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-08-25T19:55:38.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-08-25T19:55:38.306 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:38.305+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8bd860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T19:55:38.306 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:38.305+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972780 [3/bbfc30c2-dc11-4de7-ada3-fc4241ba8134] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T19:55:39.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T19:55:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-08-25T19:55:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:55:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:39.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T19:55:39.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary2 2024-08-25T19:55:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary2 2024-08-25T19:55:39.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:55:39.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T19:55:39.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-08-25T19:55:39.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:55:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T19:55:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-08-25T19:55:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:55:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:55:40.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-08-25T19:55:40.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:40.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:40.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:55:40.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:40.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:40.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:40.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:40.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:40.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T19:55:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T19:55:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b1eaf95b-5076-4fa7-856b-29a8cc179965 2024-08-25T19:55:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T19:55:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3134873.6,"bytes_per_snapshot":62697472.0,"last_snapshot_bytes":62697472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615673,"remote_snapshot_timestamp":1724615673,"replay_state":"idle"} 2024-08-25T19:55:41.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:55:41.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:55:28' 2024-08-25T19:55:41.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T19:55:41.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T19:55:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:55:41.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3134873.6,"bytes_per_snapshot":62697472.0,"last_snapshot_bytes":62697472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615673,"remote_snapshot_timestamp":1724615673,"replay_state":"idle"} 2024-08-25T19:55:41.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:55:41.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:41.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:55:41.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:41.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:55:41.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:55:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:55:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:55:41.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:41.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-08-25 19:55:28' 2024-08-25T19:55:41.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:41.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:41.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:55:41.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:41.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:55:28' 2024-08-25T19:55:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:55:41.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING841904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+replayingreplaying, {"bytes_per_second":3254681.6,"bytes_per_snapshot":65093632.0,"last_snapshot_bytes":65093632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615676,"remote_snapshot_timestamp":1724615676,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary3b1eaf95b-5076-4fa7-856b-29a8cc179965up+replayingreplaying, {"bytes_per_second":3134873.6,"bytes_per_snapshot":62697472.0,"last_snapshot_bytes":62697472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615673,"remote_snapshot_timestamp":1724615673,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+replayingreplaying, {"bytes_per_second":3342540.8,"bytes_per_snapshot":66850816.0,"last_snapshot_bytes":66850816,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615677,"remote_snapshot_timestamp":1724615677,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+replayingreplaying, {"bytes_per_second":3069337.6,"bytes_per_snapshot":61386752.0,"last_snapshot_bytes":61386752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615672,"remote_snapshot_timestamp":1724615672,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+replayingreplaying, {"bytes_per_second":3536896.0,"bytes_per_snapshot":70737920.0,"last_snapshot_bytes":70737920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+replayingreplaying, {"bytes_per_second":3507200.0,"bytes_per_snapshot":70144000.0,"last_snapshot_bytes":70144000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+replayingreplaying, {"bytes_per_second":3414016.0,"bytes_per_snapshot":68280320.0,"last_snapshot_bytes":68280320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+replayingreplaying, {"bytes_per_second":3334553.6,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28' 2024-08-25T19:55:41.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:55:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T19:55:41.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:41.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T19:55:41.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T19:55:41.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:41.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T19:55:41.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary3 2024-08-25T19:55:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:55:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:55:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-08-25T19:55:41.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-08-25T19:55:41.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:41.389+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8bc4e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T19:55:41.390 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:41.389+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972280 [3/b1eaf95b-5076-4fa7-856b-29a8cc179965] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T19:55:42.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T19:55:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-08-25T19:55:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:55:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T19:55:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary3 2024-08-25T19:55:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary3 2024-08-25T19:55:42.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:55:42.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:42.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:42.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:55:43.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-08-25T19:55:43.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:43.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:43.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:55:43.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:43.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:43.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:43.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T19:55:43.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T19:55:43.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b75218c8-fbe6-4268-8a2c-89c6f9d197fa 2024-08-25T19:55:43.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T19:55:43.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3342540.8,"bytes_per_snapshot":66850816.0,"last_snapshot_bytes":66850816,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615677,"remote_snapshot_timestamp":1724615677,"replay_state":"idle"} 2024-08-25T19:55:43.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:55:43.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:55:28' 2024-08-25T19:55:43.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T19:55:43.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T19:55:43.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:55:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3342540.8,"bytes_per_snapshot":66850816.0,"last_snapshot_bytes":66850816,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615677,"remote_snapshot_timestamp":1724615677,"replay_state":"idle"} 2024-08-25T19:55:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:55:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:55:43.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:43.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:55:43.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:55:43.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:55:44.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:55:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-08-25 19:55:28' 2024-08-25T19:55:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:55:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:55:28' 2024-08-25T19:55:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:44.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:55:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING741904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+replayingreplaying, {"bytes_per_second":3254681.6,"bytes_per_snapshot":65093632.0,"last_snapshot_bytes":65093632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615676,"remote_snapshot_timestamp":1724615676,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary4b75218c8-fbe6-4268-8a2c-89c6f9d197faup+replayingreplaying, {"bytes_per_second":3342540.8,"bytes_per_snapshot":66850816.0,"last_snapshot_bytes":66850816,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615677,"remote_snapshot_timestamp":1724615677,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+replayingreplaying, {"bytes_per_second":3069337.6,"bytes_per_snapshot":61386752.0,"last_snapshot_bytes":61386752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615672,"remote_snapshot_timestamp":1724615672,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+replayingreplaying, {"bytes_per_second":3536896.0,"bytes_per_snapshot":70737920.0,"last_snapshot_bytes":70737920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+replayingreplaying, {"bytes_per_second":3507200.0,"bytes_per_snapshot":70144000.0,"last_snapshot_bytes":70144000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+replayingreplaying, {"bytes_per_second":3414016.0,"bytes_per_snapshot":68280320.0,"last_snapshot_bytes":68280320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+replayingreplaying, {"bytes_per_second":3334553.6,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28' 2024-08-25T19:55:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:55:44.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T19:55:44.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:44.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T19:55:44.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T19:55:44.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:44.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T19:55:44.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:44.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:44.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:44.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary4 2024-08-25T19:55:44.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:55:44.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:44.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:55:44.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-08-25T19:55:44.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-08-25T19:55:44.296 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:44.295+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade2d4680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T19:55:44.296 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:44.295+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972500 [3/b75218c8-fbe6-4268-8a2c-89c6f9d197fa] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T19:55:45.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T19:55:45.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-08-25T19:55:45.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:45.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:45.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:55:45.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:45.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T19:55:45.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary4 2024-08-25T19:55:45.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:45.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary4 2024-08-25T19:55:45.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:45.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:45.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:55:46.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-08-25T19:55:46.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:46.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:46.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:55:46.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:46.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:46.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:46.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:46.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:46.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T19:55:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T19:55:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: af8e54d2-12d3-45f1-8e69-422f660f56dc 2024-08-25T19:55:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T19:55:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3069337.6,"bytes_per_snapshot":61386752.0,"last_snapshot_bytes":61386752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615672,"remote_snapshot_timestamp":1724615672,"replay_state":"idle"} 2024-08-25T19:55:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:55:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:55:28' 2024-08-25T19:55:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T19:55:46.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T19:55:46.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:55:46.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3069337.6,"bytes_per_snapshot":61386752.0,"last_snapshot_bytes":61386752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615672,"remote_snapshot_timestamp":1724615672,"replay_state":"idle"} 2024-08-25T19:55:46.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:55:46.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:46.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:55:46.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:46.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:55:46.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:55:46.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:55:46.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:55:46.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:46.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-08-25 19:55:28' 2024-08-25T19:55:46.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:46.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:46.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:55:46.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:46.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:55:28' 2024-08-25T19:55:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:55:46.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING641904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+replayingreplaying, {"bytes_per_second":3254681.6,"bytes_per_snapshot":65093632.0,"last_snapshot_bytes":65093632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615676,"remote_snapshot_timestamp":1724615676,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary5af8e54d2-12d3-45f1-8e69-422f660f56dcup+replayingreplaying, {"bytes_per_second":3069337.6,"bytes_per_snapshot":61386752.0,"last_snapshot_bytes":61386752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615672,"remote_snapshot_timestamp":1724615672,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+replayingreplaying, {"bytes_per_second":3536896.0,"bytes_per_snapshot":70737920.0,"last_snapshot_bytes":70737920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+replayingreplaying, {"bytes_per_second":3507200.0,"bytes_per_snapshot":70144000.0,"last_snapshot_bytes":70144000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+replayingreplaying, {"bytes_per_second":3414016.0,"bytes_per_snapshot":68280320.0,"last_snapshot_bytes":68280320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+replayingreplaying, {"bytes_per_second":3334553.6,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28' 2024-08-25T19:55:46.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:55:46.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T19:55:46.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:46.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T19:55:46.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T19:55:46.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:46.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T19:55:46.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary5 2024-08-25T19:55:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:55:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:55:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-08-25T19:55:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-08-25T19:55:47.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:47.095+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8af860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T19:55:47.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:47.096+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972c80 [3/af8e54d2-12d3-45f1-8e69-422f660f56dc] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T19:55:48.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T19:55:48.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-08-25T19:55:48.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:48.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:48.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:55:48.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:48.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T19:55:48.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary5 2024-08-25T19:55:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary5 2024-08-25T19:55:48.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:55:48.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T19:55:48.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-08-25T19:55:48.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:48.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:48.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:55:48.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T19:55:48.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-08-25T19:55:48.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:48.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:48.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:55:48.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:48.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:48.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:48.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:48.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:55:49.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-08-25T19:55:49.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:49.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:55:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T19:55:49.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T19:55:49.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5641da01-7c6f-4b5d-9e77-32496e6199e6 2024-08-25T19:55:49.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T19:55:49.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3536896.0,"bytes_per_snapshot":70737920.0,"last_snapshot_bytes":70737920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"} 2024-08-25T19:55:49.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:55:49.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:55:28' 2024-08-25T19:55:49.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T19:55:49.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T19:55:49.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:55:49.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3536896.0,"bytes_per_snapshot":70737920.0,"last_snapshot_bytes":70737920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"} 2024-08-25T19:55:49.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:55:49.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:49.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:55:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:55:49.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:55:49.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:55:49.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:55:49.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:49.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-08-25 19:55:28' 2024-08-25T19:55:49.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:49.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:49.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:55:49.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:49.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:55:28' 2024-08-25T19:55:49.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:49.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:49.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:55:49.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING541904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+replayingreplaying, {"bytes_per_second":3254681.6,"bytes_per_snapshot":65093632.0,"last_snapshot_bytes":65093632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615676,"remote_snapshot_timestamp":1724615676,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary65641da01-7c6f-4b5d-9e77-32496e6199e6up+replayingreplaying, {"bytes_per_second":3536896.0,"bytes_per_snapshot":70737920.0,"last_snapshot_bytes":70737920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+replayingreplaying, {"bytes_per_second":3507200.0,"bytes_per_snapshot":70144000.0,"last_snapshot_bytes":70144000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+replayingreplaying, {"bytes_per_second":3414016.0,"bytes_per_snapshot":68280320.0,"last_snapshot_bytes":68280320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+replayingreplaying, {"bytes_per_second":3334553.6,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28' 2024-08-25T19:55:49.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:55:49.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T19:55:49.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:49.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T19:55:49.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T19:55:49.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:49.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T19:55:49.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary6 2024-08-25T19:55:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:55:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:55:49.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-08-25T19:55:49.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-08-25T19:55:50.147 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:50.146+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8aeea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T19:55:50.147 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:50.146+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972f00 [3/5641da01-7c6f-4b5d-9e77-32496e6199e6] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T19:55:51.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T19:55:51.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-08-25T19:55:51.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:51.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:51.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:55:51.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T19:55:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary6 2024-08-25T19:55:51.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:51.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary6 2024-08-25T19:55:51.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:51.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:55:52.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-08-25T19:55:52.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:52.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:52.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:55:52.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:52.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:52.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:52.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:52.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:52.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T19:55:52.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T19:55:52.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 540d8938-7a8c-477e-8fc3-e65c72e78c93 2024-08-25T19:55:52.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T19:55:52.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3507200.0,"bytes_per_snapshot":70144000.0,"last_snapshot_bytes":70144000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"} 2024-08-25T19:55:52.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:55:52.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:55:28' 2024-08-25T19:55:52.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T19:55:52.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T19:55:52.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:55:52.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3507200.0,"bytes_per_snapshot":70144000.0,"last_snapshot_bytes":70144000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"} 2024-08-25T19:55:52.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:55:52.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:52.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:55:52.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:52.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:55:52.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:55:52.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:55:52.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:55:52.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:52.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-08-25 19:55:28' 2024-08-25T19:55:52.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:52.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:52.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:55:52.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:52.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:55:28' 2024-08-25T19:55:52.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:52.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:52.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:55:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING441904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+replayingreplaying, {"bytes_per_second":3254681.6,"bytes_per_snapshot":65093632.0,"last_snapshot_bytes":65093632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615676,"remote_snapshot_timestamp":1724615676,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary7540d8938-7a8c-477e-8fc3-e65c72e78c93up+replayingreplaying, {"bytes_per_second":3507200.0,"bytes_per_snapshot":70144000.0,"last_snapshot_bytes":70144000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+replayingreplaying, {"bytes_per_second":3414016.0,"bytes_per_snapshot":68280320.0,"last_snapshot_bytes":68280320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+replayingreplaying, {"bytes_per_second":3334553.6,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28' 2024-08-25T19:55:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:55:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T19:55:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T19:55:52.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T19:55:52.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:52.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T19:55:52.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:52.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:52.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:52.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary7 2024-08-25T19:55:52.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:55:52.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:52.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:55:52.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-08-25T19:55:52.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-08-25T19:55:53.097 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:53.096+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade6a1520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T19:55:53.097 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:53.096+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973180 [3/540d8938-7a8c-477e-8fc3-e65c72e78c93] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T19:55:54.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T19:55:54.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-08-25T19:55:54.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:54.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:54.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:55:54.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:54.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T19:55:54.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary7 2024-08-25T19:55:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary7 2024-08-25T19:55:54.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:55:54.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T19:55:54.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-08-25T19:55:54.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:54.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:54.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:55:54.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T19:55:54.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-08-25T19:55:54.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:54.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:54.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:55:54.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:54.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:54.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:54.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:54.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:55:55.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-08-25T19:55:55.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:55.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:55.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:55:55.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:55.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:55.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:55.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:55.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:55.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T19:55:55.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T19:55:55.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5fdd517d-317f-4c25-b8f4-f5395bd911d6 2024-08-25T19:55:55.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T19:55:55.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3414016.0,"bytes_per_snapshot":68280320.0,"last_snapshot_bytes":68280320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"} 2024-08-25T19:55:55.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:55:55.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:55:28' 2024-08-25T19:55:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T19:55:55.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T19:55:55.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:55:55.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3414016.0,"bytes_per_snapshot":68280320.0,"last_snapshot_bytes":68280320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"} 2024-08-25T19:55:55.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:55:55.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:55.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:55:55.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:55.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:55:55.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:55:55.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:55:55.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:55:55.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:55.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-08-25 19:55:28' 2024-08-25T19:55:55.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:55.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:55.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:55:55.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:55.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:55:28' 2024-08-25T19:55:55.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:55.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:55.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:55:55.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING341904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+replayingreplaying, {"bytes_per_second":3254681.6,"bytes_per_snapshot":65093632.0,"last_snapshot_bytes":65093632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615676,"remote_snapshot_timestamp":1724615676,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary85fdd517d-317f-4c25-b8f4-f5395bd911d6up+replayingreplaying, {"bytes_per_second":3414016.0,"bytes_per_snapshot":68280320.0,"last_snapshot_bytes":68280320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615674,"remote_snapshot_timestamp":1724615674,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+replayingreplaying, {"bytes_per_second":3334553.6,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:28' 2024-08-25T19:55:55.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:55:55.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T19:55:55.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:28' 2024-08-25T19:55:55.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T19:55:55.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T19:55:55.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:55.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T19:55:55.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:55.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:55.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:55.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary8 2024-08-25T19:55:55.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:55:55.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:55.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:55:55.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-08-25T19:55:55.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-08-25T19:55:56.082 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:56.081+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8afba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T19:55:56.083 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:56.081+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973900 [3/5fdd517d-317f-4c25-b8f4-f5395bd911d6] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T19:55:57.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T19:55:57.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-08-25T19:55:57.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:57.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:57.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:55:57.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:57.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T19:55:57.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary8 2024-08-25T19:55:57.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:57.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary8 2024-08-25T19:55:57.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:55:57.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:55:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:55:58.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-08-25T19:55:58.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:58.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:58.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:55:58.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:58.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:55:58.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:55:58.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:58.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:58.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T19:55:58.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T19:55:58.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 73264916-cfd5-40d0-83ae-ff7d92659663 2024-08-25T19:55:58.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T19:55:58.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"} 2024-08-25T19:55:58.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:55:58.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:55:58' 2024-08-25T19:55:58.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T19:55:58.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T19:55:58.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:55:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"} 2024-08-25T19:55:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:55:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:55:59.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:59.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:55:59.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:55:59.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:55:59.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:55:59.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:58' 2024-08-25T19:55:59.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-08-25 19:55:58' 2024-08-25T19:55:59.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:55:59.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:59.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:55:59.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:55:59.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:55:58' 2024-08-25T19:55:59.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:55:59.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:55:59.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:55:59.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING241904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":65093632.0,"last_snapshot_bytes":65093632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615676,"remote_snapshot_timestamp":1724615676,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:58image-primary973264916-cfd5-40d0-83ae-ff7d92659663up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615675,"remote_snapshot_timestamp":1724615675,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:58' 2024-08-25T19:55:59.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:55:59.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T19:55:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:58' 2024-08-25T19:55:59.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T19:55:59.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T19:55:59.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:55:59.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T19:55:59.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:55:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:55:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary9 2024-08-25T19:55:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:55:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:55:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:55:59.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-08-25T19:55:59.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-08-25T19:55:59.296 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:59.294+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade2d7380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T19:55:59.296 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:55:59.294+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973b80 [3/73264916-cfd5-40d0-83ae-ff7d92659663] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T19:56:00.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T19:56:00.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-08-25T19:56:00.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:00.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:00.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:00.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:56:00.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T19:56:00.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary9 2024-08-25T19:56:00.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:00.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary9 2024-08-25T19:56:00.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:56:00.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T19:56:00.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-08-25T19:56:00.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:00.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:00.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:00.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T19:56:00.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-08-25T19:56:00.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:00.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:00.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:00.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:56:00.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:56:00.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:56:00.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:56:00.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T19:56:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-08-25T19:56:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:01.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:56:01.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T19:56:01.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T19:56:01.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:56:01.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:56:01.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T19:56:01.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T19:56:01.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: fad93bc2-e996-49cb-a574-393a1cbbe56a 2024-08-25T19:56:01.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T19:56:01.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":65093632.0,"last_snapshot_bytes":65093632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615676,"remote_snapshot_timestamp":1724615676,"replay_state":"idle"} 2024-08-25T19:56:01.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T19:56:01.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 19:55:58' 2024-08-25T19:56:01.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T19:56:01.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T19:56:01.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T19:56:01.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":65093632.0,"last_snapshot_bytes":65093632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615676,"remote_snapshot_timestamp":1724615676,"replay_state":"idle"} 2024-08-25T19:56:01.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T19:56:01.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:56:01.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T19:56:01.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:56:01.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T19:56:01.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T19:56:01.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T19:56:01.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T19:56:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:58' 2024-08-25T19:56:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-08-25 19:55:58' 2024-08-25T19:56:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T19:56:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 19:55:58' 2024-08-25T19:56:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T19:56:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T19:56:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T19:56:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING141904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10fad93bc2-e996-49cb-a574-393a1cbbe56aup+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":65093632.0,"last_snapshot_bytes":65093632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724615676,"remote_snapshot_timestamp":1724615676,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 19:55:58' 2024-08-25T19:56:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T19:56:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T19:56:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 19:55:58' 2024-08-25T19:56:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T19:56:01.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T19:56:01.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T19:56:01.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T19:56:01.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T19:56:01.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:01.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary10 2024-08-25T19:56:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T19:56:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-08-25T19:56:01.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-08-25T19:56:02.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:56:02.069+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8af380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T19:56:02.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T19:56:02.069+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555adf432000 [3/fad93bc2-e996-49cb-a574-393a1cbbe56a] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T19:56:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T19:56:03.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-08-25T19:56:03.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:03.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:03.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:03.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T19:56:03.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T19:56:03.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary10 2024-08-25T19:56:03.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:03.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary10 2024-08-25T19:56:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..10} 2024-08-25T19:56:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS=() 2024-08-25T19:56:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS=() 2024-08-25T19:56:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS=() 2024-08-25T19:56:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS=() 2024-08-25T19:56:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RET=0 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary1 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-08-25T19:56:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:56:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:56:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:56:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-08-25T19:56:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:56:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:56:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-08-25T19:56:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-08-25T19:56:05.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:56:05.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-08-25T19:56:05.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary1': 2024-08-25T19:56:05.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:56:05.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:56:05.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:56:05.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 15a2977bcaf0 2024-08-25T19:56:05.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.15a2977bcaf0 2024-08-25T19:56:05.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:56:05.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:56:05.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:56:05.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:56:05.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:56:03 2024 2024-08-25T19:56:05.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:56:03 2024 2024-08-25T19:56:05.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:56:03 2024 2024-08-25T19:56:05.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:56:05.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:56:05.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 4c2ba3a4-f815-4f35-86fb-0f609016cfb2 2024-08-25T19:56:05.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:56:05.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:56:05.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:56:05.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-08-25T19:56:05.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd0 2024-08-25T19:56:05.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:56:05.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-08-25T19:56:05.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:56:05.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:56:05.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:56:05.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 1531f35b-c99a-42ce-a43d-dbc8f0381fbc 2024-08-25T19:56:05.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:56:05.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:56:05.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:05.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:56:05.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:56:05.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:56:05.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:56:05.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:05.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary1 2024-08-25T19:56:05.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-08-25T19:56:05.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:56:05.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary1 2024-08-25T19:56:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:56:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-08-25T19:56:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:56:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:56:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:56:05.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:56:05.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T19:56:05.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary2 2024-08-25T19:56:05.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary1 2024-08-25T19:56:05.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-08-25T19:56:05.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:05.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:05.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:05.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:05.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary1 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:56:05.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:56:05.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:56:05.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:05.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-08-25T19:56:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:56:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:56:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-08-25T19:56:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:05.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-08-25T19:56:05.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:56:05.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-08-25T19:56:06.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:56:06.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 334 2024-08-25T19:56:06.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:06.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-08-25T19:56:06.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary2': 2024-08-25T19:56:06.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:56:06.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:56:06.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:56:06.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 15b249233b92 2024-08-25T19:56:06.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.15b249233b92 2024-08-25T19:56:06.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:56:06.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:56:06.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:56:06.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:56:06.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:56:05 2024 2024-08-25T19:56:06.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:56:05 2024 2024-08-25T19:56:06.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:56:05 2024 2024-08-25T19:56:06.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:56:06.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:56:06.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 4b479bdf-9f6e-4198-8e79-ff93a2d21c04 2024-08-25T19:56:06.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:56:06.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:56:06.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:56:06.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-08-25T19:56:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd1 2024-08-25T19:56:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:56:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-08-25T19:56:06.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:56:06.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:56:06.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:56:06.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 75eb8f23-617c-40eb-8722-533fbcd8773b 2024-08-25T19:56:06.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:56:06.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:56:06.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:06.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:56:06.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:56:06.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:56:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:56:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:06.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary2 2024-08-25T19:56:06.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-08-25T19:56:06.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:56:06.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary2 2024-08-25T19:56:06.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:56:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary2 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary3 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary2 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-08-25T19:56:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:56:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:56:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:56:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-08-25T19:56:06.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:56:06.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:06.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:56:06.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:06.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-08-25T19:56:06.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:06.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:06.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:06.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:06.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-08-25T19:56:06.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:56:06.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-08-25T19:56:07.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:56:07.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 336 2024-08-25T19:56:07.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-08-25T19:56:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary3': 2024-08-25T19:56:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:56:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:56:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:56:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 15c0459f1901 2024-08-25T19:56:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.15c0459f1901 2024-08-25T19:56:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:56:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:56:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:56:07.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:56:07.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:56:06 2024 2024-08-25T19:56:07.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:56:06 2024 2024-08-25T19:56:07.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:56:06 2024 2024-08-25T19:56:07.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:56:07.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:56:07.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 6a6a0d98-0015-4435-a50c-afd86813e19e 2024-08-25T19:56:07.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:56:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:56:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:56:07.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-08-25T19:56:07.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd2 2024-08-25T19:56:07.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:56:07.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-08-25T19:56:07.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:56:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:56:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:56:07.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 365a7f19-71f5-44fd-a701-663e2837f344 2024-08-25T19:56:07.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:56:07.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:56:07.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:07.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:56:07.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:56:07.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:56:07.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:56:07.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary3 2024-08-25T19:56:07.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-08-25T19:56:07.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:56:07.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary3 2024-08-25T19:56:07.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:56:07.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T19:56:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary4 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary3 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary3 2024-08-25T19:56:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:07.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:07.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-08-25T19:56:07.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:07.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:07.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:07.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:07.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:56:07.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:56:07.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:56:07.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:07.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-08-25T19:56:07.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:56:07.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:07.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:56:07.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:07.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-08-25T19:56:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-08-25T19:56:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:56:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-08-25T19:56:08.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:56:08.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 338 2024-08-25T19:56:08.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:08.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-08-25T19:56:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary4': 2024-08-25T19:56:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:56:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:56:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:56:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 15cfe3f90a6e 2024-08-25T19:56:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.15cfe3f90a6e 2024-08-25T19:56:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:56:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:56:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:56:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:56:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:56:07 2024 2024-08-25T19:56:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:56:07 2024 2024-08-25T19:56:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:56:07 2024 2024-08-25T19:56:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:56:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:56:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 5f5a643b-a56b-4251-aa94-d881dc6cc262 2024-08-25T19:56:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:56:08.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:56:08.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:56:08.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-08-25T19:56:08.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd3 2024-08-25T19:56:08.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:56:08.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-08-25T19:56:08.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:56:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:56:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:56:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 96645e59-75aa-4589-a63c-6beb792fef9f 2024-08-25T19:56:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:56:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:56:08.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:08.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:56:08.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:56:08.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:56:08.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:56:08.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:08.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary4 2024-08-25T19:56:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-08-25T19:56:08.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:56:08.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary4 2024-08-25T19:56:08.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:56:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-08-25T19:56:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:56:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:56:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:56:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T19:56:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary5 2024-08-25T19:56:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-08-25T19:56:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary4 2024-08-25T19:56:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:56:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary4 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:56:08.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:56:08.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:08.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-08-25T19:56:08.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:56:08.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:08.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:56:08.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:08.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-08-25T19:56:08.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:08.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:08.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:08.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:08.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-08-25T19:56:08.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:56:08.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-08-25T19:56:09.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:56:09.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 340 2024-08-25T19:56:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:09.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-08-25T19:56:09.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:09.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:56:09.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:09.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:09.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:09.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:56:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary5': 2024-08-25T19:56:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:56:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:56:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:56:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 15dec346906e 2024-08-25T19:56:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.15dec346906e 2024-08-25T19:56:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:56:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:56:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:56:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:56:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:56:08 2024 2024-08-25T19:56:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:56:08 2024 2024-08-25T19:56:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:56:08 2024 2024-08-25T19:56:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:56:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:56:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: d07b684d-ae92-495b-af63-7a0de0e70d08 2024-08-25T19:56:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:56:09.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:56:09.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:56:09.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-08-25T19:56:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd4 2024-08-25T19:56:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:56:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-08-25T19:56:09.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:56:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:56:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:56:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 987554ba-c628-4980-a4d2-8254b96a313d 2024-08-25T19:56:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:56:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:56:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:56:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:56:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:56:09.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:56:09.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:09.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary5 2024-08-25T19:56:09.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-08-25T19:56:09.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:56:09.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary5 2024-08-25T19:56:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:56:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-08-25T19:56:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:56:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:56:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T19:56:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary6 2024-08-25T19:56:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary5 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:56:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary5 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-08-25T19:56:09.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-08-25T19:56:09.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:56:09.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:09.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:56:09.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:09.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-08-25T19:56:09.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:09.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:09.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:09.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:09.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-08-25T19:56:09.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:56:09.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-08-25T19:56:10.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 343 2024-08-25T19:56:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:56:10.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 342 2024-08-25T19:56:10.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:10.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:10.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-08-25T19:56:10.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:10.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:56:10.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:10.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:10.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:10.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:56:10.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary6': 2024-08-25T19:56:10.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:56:10.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:56:10.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 15f0e7064484 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.15f0e7064484 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:56:09 2024 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:56:09 2024 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:56:09 2024 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: ba93f24c-de3f-4b9f-9f95-a8a9c5aa159d 2024-08-25T19:56:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:56:10.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:56:10.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:56:10.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-08-25T19:56:10.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd5 2024-08-25T19:56:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:56:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-08-25T19:56:10.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:56:10.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:56:10.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:56:10.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 321497b8-e265-471f-b5ab-82a0814e1957 2024-08-25T19:56:10.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:56:10.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:56:10.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:10.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:56:10.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:56:10.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:56:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:56:10.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:10.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary6 2024-08-25T19:56:10.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-08-25T19:56:10.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:56:10.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary6 2024-08-25T19:56:10.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:56:10.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-08-25T19:56:10.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:10.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:10.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:10.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:10.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:56:10.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:10.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary7 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary6 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:56:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:56:10.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:10.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary6 2024-08-25T19:56:10.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-08-25T19:56:10.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-08-25T19:56:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:56:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:56:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-08-25T19:56:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-08-25T19:56:10.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:56:10.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-08-25T19:56:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:56:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 346 2024-08-25T19:56:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 345 2024-08-25T19:56:11.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:11.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:11.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-08-25T19:56:11.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:11.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:56:11.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:11.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:11.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:11.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:56:11.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary7': 2024-08-25T19:56:11.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:56:11.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:56:11.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:56:11.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1602a0bba459 2024-08-25T19:56:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1602a0bba459 2024-08-25T19:56:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:56:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:56:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:56:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:56:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:56:10 2024 2024-08-25T19:56:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:56:10 2024 2024-08-25T19:56:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:56:10 2024 2024-08-25T19:56:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:56:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:56:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 12419548-8f80-4684-838b-1b7f43caadcb 2024-08-25T19:56:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:56:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:56:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:56:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-08-25T19:56:11.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd6 2024-08-25T19:56:11.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:56:11.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-08-25T19:56:11.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:56:11.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:56:11.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:56:11.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 83e6da50-a0be-4364-9419-14d4f48f1e48 2024-08-25T19:56:11.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:56:11.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:56:11.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:11.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:56:11.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:56:11.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:56:11.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:56:11.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:11.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary7 2024-08-25T19:56:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-08-25T19:56:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:56:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary7 2024-08-25T19:56:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:56:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-08-25T19:56:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:56:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:56:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary8 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary7 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:56:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary7 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-08-25T19:56:11.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-08-25T19:56:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:56:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:56:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-08-25T19:56:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-08-25T19:56:11.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:56:11.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-08-25T19:56:12.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:12.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:56:12.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:12.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:12.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:12.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:56:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:56:12.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 348 2024-08-25T19:56:12.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 349 2024-08-25T19:56:12.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-08-25T19:56:12.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:12.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:12.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary8': 2024-08-25T19:56:12.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:56:12.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:56:12.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:56:12.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 16145fbd849f 2024-08-25T19:56:12.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.16145fbd849f 2024-08-25T19:56:12.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:56:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:56:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:56:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:56:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:56:11 2024 2024-08-25T19:56:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:56:11 2024 2024-08-25T19:56:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:56:11 2024 2024-08-25T19:56:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:56:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:56:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 6836838d-c369-4977-8900-df8dc7a72cf6 2024-08-25T19:56:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:56:12.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:56:12.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:56:12.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-08-25T19:56:12.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd7 2024-08-25T19:56:12.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:56:12.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-08-25T19:56:12.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:56:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:56:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:56:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 26c0b60f-607f-484a-a37c-9027e0ac1ba8 2024-08-25T19:56:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:56:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:56:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:12.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:56:12.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:56:12.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:56:12.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:56:12.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:12.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary8 2024-08-25T19:56:12.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-08-25T19:56:12.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:56:12.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary8 2024-08-25T19:56:12.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:56:12.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-08-25T19:56:12.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:12.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:12.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:12.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:12.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary9 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary8 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary8 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:56:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:56:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-08-25T19:56:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:56:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-08-25T19:56:12.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:56:12.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:12.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:56:12.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-08-25T19:56:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-08-25T19:56:12.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:56:12.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-08-25T19:56:13.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:13.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:56:13.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:13.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:13.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:13.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:56:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:56:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:13.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:13.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:56:13.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:56:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 352 2024-08-25T19:56:13.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 351 2024-08-25T19:56:13.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-08-25T19:56:13.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:13.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:13.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary9': 2024-08-25T19:56:13.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:56:13.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:56:13.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 16262c7e08c8 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.16262c7e08c8 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:56:12 2024 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:56:12 2024 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:56:12 2024 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 44ece9fc-ce13-4f35-b342-6d5322202441 2024-08-25T19:56:13.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:56:13.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:56:13.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:56:13.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-08-25T19:56:13.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd8 2024-08-25T19:56:13.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:56:13.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-08-25T19:56:13.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:56:13.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:56:13.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:56:13.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 1a11ea9d-bd27-40af-a8a4-443253d595c2 2024-08-25T19:56:13.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:56:13.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:56:13.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:13.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:56:13.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:56:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:56:13.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:56:13.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary9 2024-08-25T19:56:13.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-08-25T19:56:13.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:56:13.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary9 2024-08-25T19:56:13.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:56:13.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-08-25T19:56:13.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary10 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary9 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:13.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary9 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-08-25T19:56:13.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:13.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:13.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T19:56:13.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T19:56:13.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T19:56:13.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T19:56:13.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-08-25T19:56:13.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T19:56:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T19:56:13.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T19:56:13.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-08-25T19:56:13.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:13.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:13.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:13.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T19:56:13.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-08-25T19:56:13.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:56:13.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-08-25T19:56:14.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:14.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:56:14.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:14.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:14.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:14.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:56:14.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:14.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:56:14.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:14.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:14.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:14.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:56:14.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 355 2024-08-25T19:56:14.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T19:56:14.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:14.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 356 2024-08-25T19:56:14.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 354 2024-08-25T19:56:14.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:14.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:14.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-08-25T19:56:14.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary10': 2024-08-25T19:56:14.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T19:56:14.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T19:56:14.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T19:56:14.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 163bf35fd819 2024-08-25T19:56:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.163bf35fd819 2024-08-25T19:56:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T19:56:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T19:56:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T19:56:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T19:56:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 19:56:13 2024 2024-08-25T19:56:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 19:56:13 2024 2024-08-25T19:56:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 19:56:13 2024 2024-08-25T19:56:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T19:56:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T19:56:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 0f6047f7-e198-4c21-aeb1-fcc11903a8a7 2024-08-25T19:56:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T19:56:14.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T19:56:14.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T19:56:14.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-08-25T19:56:14.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd9 2024-08-25T19:56:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T19:56:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-08-25T19:56:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T19:56:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T19:56:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T19:56:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 73864998-85b4-42a9-a8a1-02b4e60c694a 2024-08-25T19:56:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T19:56:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T19:56:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:14.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T19:56:14.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T19:56:14.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T19:56:14.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T19:56:14.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T19:56:14.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary10 2024-08-25T19:56:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-08-25T19:56:14.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T19:56:14.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary10 2024-08-25T19:56:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T19:56:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-08-25T19:56:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:56:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:56:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T19:56:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:56:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 35251 2024-08-25T19:56:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary10 2024-08-25T19:56:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary10 2024-08-25T19:56:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-08-25T19:56:14.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T19:56:14.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-08-25T19:56:15.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:15.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:56:15.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:15.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:15.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:15.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:56:15.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:15.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:56:15.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:15.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:15.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:15.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:56:15.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 359 2024-08-25T19:56:15.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 358 2024-08-25T19:56:15.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 360 2024-08-25T19:56:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:15.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:16.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 362 2024-08-25T19:56:16.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 361 2024-08-25T19:56:16.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:16.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:16.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:16.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:56:16.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:16.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:16.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:16.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:56:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:56:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:56:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 364 2024-08-25T19:56:17.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 363 2024-08-25T19:56:17.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:17.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:17.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:17.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:56:17.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:17.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:17.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:17.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:56:17.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:17.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:17.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:56:17.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:56:17.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:17.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:17.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:17.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:17.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:17.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:17.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:56:17.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:56:18.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 367 2024-08-25T19:56:18.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 365 2024-08-25T19:56:18.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 366 2024-08-25T19:56:18.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:18.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:18.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:18.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:18.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:56:18.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:18.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:18.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:18.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:56:18.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:18.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:56:18.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:18.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:18.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:18.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:56:18.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:18.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:56:18.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:18.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:18.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:18.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:56:19.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 370 2024-08-25T19:56:19.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:19.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:56:19.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:19.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:19.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:19.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:56:19.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:19.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:56:19.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:19.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:19.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:19.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:56:19.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 369 2024-08-25T19:56:19.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 368 2024-08-25T19:56:19.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:19.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:19.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:20.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:20.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:56:20.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:20.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:20.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:20.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:56:20.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:20.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:56:20.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:20.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:20.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:20.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:56:20.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 372 2024-08-25T19:56:20.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 371 2024-08-25T19:56:20.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:20.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:21.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 374 2024-08-25T19:56:21.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 373 2024-08-25T19:56:21.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:21.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:56:21.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:21.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:21.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:21.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:56:21.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:21.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:56:21.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:21.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:21.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:21.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:56:21.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:21.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:56:21.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:21.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:21.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:21.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:56:21.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:21.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:22.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:22.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:56:22.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:22.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:22.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:22.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:56:22.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:22.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:56:22.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:22.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:22.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:22.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:56:22.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:22.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:56:22.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:22.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:22.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:22.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:56:22.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 375 2024-08-25T19:56:22.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:22.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 376 2024-08-25T19:56:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:22.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 377 2024-08-25T19:56:22.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:56:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:56:23.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:23.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:56:23.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:23.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:23.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:23.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:56:23.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 379 2024-08-25T19:56:23.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 380 2024-08-25T19:56:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:23.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:23.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 378 2024-08-25T19:56:23.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:24.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:24.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:56:24.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:24.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:24.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:24.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:56:24.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:24.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:56:24.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:24.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:24.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:24.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:56:24.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 382 2024-08-25T19:56:24.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:24.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 381 2024-08-25T19:56:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 383 2024-08-25T19:56:25.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:25.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 384 2024-08-25T19:56:25.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:25.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:56:25.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:25.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:25.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:25.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:56:25.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:25.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:25.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:56:25.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:25.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:25.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:25.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:56:25.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:25.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:56:25.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:25.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:25.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:25.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:56:26.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 386 2024-08-25T19:56:26.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:26.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 385 2024-08-25T19:56:26.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:26.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:26.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:56:26.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:26.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:26.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:26.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:56:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:56:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:56:26.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:26.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:56:26.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:26.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:26.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:26.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:56:27.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 388 2024-08-25T19:56:27.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 389 2024-08-25T19:56:27.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:27.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:56:27.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:27.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:27.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:27.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:56:27.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:27.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:27.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:27.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:56:27.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:27.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:27.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:27.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:56:28.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 387 2024-08-25T19:56:28.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:28.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:28.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:56:28.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:28.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:28.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:28.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:56:28.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:28.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:56:28.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:28.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:28.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:28.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:56:28.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 392 2024-08-25T19:56:28.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:29.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 390 2024-08-25T19:56:29.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:29.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:29.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:56:29.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:29.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:29.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:29.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:56:29.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:29.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:56:29.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:29.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:29.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:29.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:56:29.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 393 2024-08-25T19:56:29.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:30.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 391 2024-08-25T19:56:30.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:30.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:30.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:56:30.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:30.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:30.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:30.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:56:30.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:30.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:56:30.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:30.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:30.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:30.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:56:30.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 395 2024-08-25T19:56:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:31.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 394 2024-08-25T19:56:31.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:31.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:56:31.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:31.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:31.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:31.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:56:31.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:31.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:31.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:56:31.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:31.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:31.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:31.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:56:31.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 397 2024-08-25T19:56:31.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:32.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 396 2024-08-25T19:56:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:56:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:56:32.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:32.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 399 2024-08-25T19:56:32.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:32.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:32.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:56:32.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:32.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:32.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:32.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:56:33.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 398 2024-08-25T19:56:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:56:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:56:33.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:33.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 401 2024-08-25T19:56:33.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:33.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:33.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:56:33.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:33.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:33.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:33.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:56:34.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 400 2024-08-25T19:56:34.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:34.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:34.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:56:34.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:34.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:34.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:34.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:56:34.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 403 2024-08-25T19:56:34.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:34.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:34.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:56:34.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:34.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:34.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:34.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:56:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 402 2024-08-25T19:56:35.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:35.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:35.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:56:35.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:35.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:35.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:35.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:56:35.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:35.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:56:35.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:35.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:35.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:35.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:56:35.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 405 2024-08-25T19:56:35.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:36.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 404 2024-08-25T19:56:36.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:36.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:36.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:56:36.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:36.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:36.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:36.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:56:36.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 408 2024-08-25T19:56:36.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:36.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:56:36.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:36.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:36.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:36.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:56:36.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:37.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 406 2024-08-25T19:56:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:56:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:56:37.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:37.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:37.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:56:37.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:37.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:37.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:37.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:56:37.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 409 2024-08-25T19:56:37.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:38.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 407 2024-08-25T19:56:38.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:38.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:56:38.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:38.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:38.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:38.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:56:38.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:56:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:56:38.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 411 2024-08-25T19:56:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:39.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:39.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:56:39.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:39.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:39.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:39.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:56:39.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 410 2024-08-25T19:56:39.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:39.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:56:39.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:39.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:39.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:39.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:56:39.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:39.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 413 2024-08-25T19:56:39.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:40.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 412 2024-08-25T19:56:40.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:40.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:40.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:56:40.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:40.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:40.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:40.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:56:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 416 2024-08-25T19:56:40.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:56:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:56:41.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 414 2024-08-25T19:56:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:41.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 417 2024-08-25T19:56:41.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:41.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:56:41.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:41.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:41.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:41.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:56:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:56:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:56:42.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 415 2024-08-25T19:56:42.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:42.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:42.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:56:42.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:42.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:42.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:42.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:56:42.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 419 2024-08-25T19:56:42.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:42.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:42.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:56:42.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:42.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:42.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:42.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:56:43.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 418 2024-08-25T19:56:43.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:43.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:56:43.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:43.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:43.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:43.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:56:43.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:43.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 421 2024-08-25T19:56:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:56:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:56:43.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:56:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:56:44.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 420 2024-08-25T19:56:44.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:44.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:44.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:56:44.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:44.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:44.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:44.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:56:44.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 423 2024-08-25T19:56:44.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:45.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 422 2024-08-25T19:56:45.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:45.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:56:45.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:45.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:45.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:45.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:56:45.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:45.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 425 2024-08-25T19:56:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:56:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:45.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:56:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:46.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 424 2024-08-25T19:56:46.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:46.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:46.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:56:46.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:46.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:46.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:46.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:56:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 427 2024-08-25T19:56:46.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:46.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:56:46.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:46.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:46.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:46.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:56:46.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:47.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 426 2024-08-25T19:56:47.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:47.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:56:47.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:47.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:47.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:47.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:56:47.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:47.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 429 2024-08-25T19:56:47.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:47.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:56:47.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:47.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:47.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:47.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:56:47.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:48.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 428 2024-08-25T19:56:48.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:48.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:56:48.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:48.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:48.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:48.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:56:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:48.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 431 2024-08-25T19:56:48.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:48.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:48.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:56:48.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:48.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:48.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:48.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:56:49.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:49.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:56:49.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:49.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:49.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:49.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:56:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 430 2024-08-25T19:56:49.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:49.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 433 2024-08-25T19:56:49.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:49.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:49.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:56:49.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:49.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:49.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:49.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:56:50.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 432 2024-08-25T19:56:50.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:50.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:50.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:56:50.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:50.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:50.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:50.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:56:50.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 435 2024-08-25T19:56:50.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:56:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:50.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:50.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:56:51.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 434 2024-08-25T19:56:51.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:51.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 437 2024-08-25T19:56:51.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:51.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:51.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:56:51.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:51.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:51.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:51.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:56:51.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:51.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:56:51.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:51.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:51.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:51.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:56:52.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 436 2024-08-25T19:56:52.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 439 2024-08-25T19:56:52.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:52.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:52.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:56:52.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:52.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:52.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:52.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:56:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:56:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:56:53.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 438 2024-08-25T19:56:53.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:53.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 441 2024-08-25T19:56:53.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:53.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:56:53.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:53.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:53.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:53.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:56:53.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:53.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:53.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:56:53.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:53.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:53.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:53.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:56:54.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 440 2024-08-25T19:56:54.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:54.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:56:54.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:54.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:54.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:54.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:56:54.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:54.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:54.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:56:54.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:54.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:54.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:56:54.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:56:54.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 443 2024-08-25T19:56:54.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:55.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 442 2024-08-25T19:56:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:56:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:56:55.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:55.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:55.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:56:55.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:55.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:55.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:56:55.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:56:55.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 446 2024-08-25T19:56:55.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:56.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 444 2024-08-25T19:56:56.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:56.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 447 2024-08-25T19:56:56.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:56.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:56:56.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:56.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:56.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:56:56.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:56:56.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:56.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:56.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:56:56.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:56.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:56.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:56:56.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:56:57.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 445 2024-08-25T19:56:57.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:57.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:56:57.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:57.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:57.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:56:57.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:56:57.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:57.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 449 2024-08-25T19:56:57.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:57.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:57.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:56:57.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:57.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:57.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:56:57.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:56:58.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 448 2024-08-25T19:56:58.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:58.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:58.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:56:58.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:58.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:58.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:56:58.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:56:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 451 2024-08-25T19:56:58.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:58.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:58.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:56:58.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:58.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:58.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:56:58.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:56:59.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 450 2024-08-25T19:56:59.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:59.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:56:59.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:59.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:59.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:56:59.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:56:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:56:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:56:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:56:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:56:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:56:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:56:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:56:59.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 453 2024-08-25T19:56:59.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:00.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:00.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:57:00.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:00.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:00.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:57:00.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:57:00.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 452 2024-08-25T19:57:00.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:57:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:57:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:57:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 455 2024-08-25T19:57:00.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 454 2024-08-25T19:57:01.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:01.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 457 2024-08-25T19:57:01.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:57:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:57:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:57:01.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:01.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:57:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:57:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:57:02.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 456 2024-08-25T19:57:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:57:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:57:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:57:02.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:57:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:57:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:57:02.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 459 2024-08-25T19:57:02.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 458 2024-08-25T19:57:03.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:03.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:03.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:57:03.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:03.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:03.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:57:03.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:57:03.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 461 2024-08-25T19:57:03.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:03.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:03.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:57:03.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:03.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:03.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:57:03.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:57:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:57:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:04.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:57:04.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:57:04.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 460 2024-08-25T19:57:04.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:04.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:57:04.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:57:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:57:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:04.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 463 2024-08-25T19:57:04.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:05.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 462 2024-08-25T19:57:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:57:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:57:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:57:05.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 465 2024-08-25T19:57:05.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:05.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:57:05.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:05.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:05.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:57:05.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:57:05.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:06.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 464 2024-08-25T19:57:06.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:06.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 467 2024-08-25T19:57:06.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:06.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:06.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:57:06.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:06.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:06.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:57:06.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:57:06.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:06.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:57:06.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:57:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:57:07.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 466 2024-08-25T19:57:07.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:07.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:57:07.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:07.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:07.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:57:07.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:57:07.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:07.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 469 2024-08-25T19:57:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:57:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:57:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:57:08.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 468 2024-08-25T19:57:08.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:08.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:57:08.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:08.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:08.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:57:08.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:57:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:08.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 471 2024-08-25T19:57:08.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:08.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:57:08.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:08.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:08.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:57:08.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:57:08.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:09.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:57:09.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:09.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:09.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:57:09.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:57:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:57:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:09.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:09.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:57:09.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:57:09.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 470 2024-08-25T19:57:09.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:09.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 473 2024-08-25T19:57:09.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:10.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 472 2024-08-25T19:57:10.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:10.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:57:10.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:10.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:10.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:57:10.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:57:10.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:10.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:10.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:57:10.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:10.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:10.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:57:10.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:57:10.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 476 2024-08-25T19:57:10.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:11.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 474 2024-08-25T19:57:11.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:11.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 477 2024-08-25T19:57:11.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:57:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:57:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:57:11.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:11.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:57:11.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:11.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:11.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:57:11.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:57:12.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 475 2024-08-25T19:57:12.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:12.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 479 2024-08-25T19:57:12.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:12.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:12.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:57:12.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:12.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:12.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:57:12.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:57:12.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:12.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:57:12.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:12.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:12.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:57:12.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:57:13.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 478 2024-08-25T19:57:13.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:13.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 481 2024-08-25T19:57:13.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:13.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:13.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:57:13.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:13.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:13.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:57:13.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:57:13.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:13.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:57:13.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:13.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:13.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:57:13.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:57:14.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 480 2024-08-25T19:57:14.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:14.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:57:14.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:14.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:14.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:57:14.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:57:14.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:14.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:57:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:57:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:57:14.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 483 2024-08-25T19:57:14.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:15.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 482 2024-08-25T19:57:15.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:15.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:15.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:57:15.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:15.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:15.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:57:15.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:57:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 485 2024-08-25T19:57:15.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:15.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:15.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:57:15.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:15.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:15.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:57:15.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:57:16.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 484 2024-08-25T19:57:16.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:16.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:57:16.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:16.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:16.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:57:16.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:57:16.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:16.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 487 2024-08-25T19:57:16.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:16.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:57:16.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:16.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:16.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:57:16.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:57:16.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 486 2024-08-25T19:57:17.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:17.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:57:17.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:17.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:17.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:57:17.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:57:17.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:17.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 489 2024-08-25T19:57:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:57:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:57:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:57:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:18.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 488 2024-08-25T19:57:18.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:18.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:57:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:57:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:57:18.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:18.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 491 2024-08-25T19:57:18.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:18.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:57:18.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:18.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:18.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:57:18.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:57:18.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:19.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 490 2024-08-25T19:57:19.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:19.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:57:19.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:19.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:19.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:57:19.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:57:19.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:19.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:19.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:57:19.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:19.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:19.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:57:19.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:57:19.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 493 2024-08-25T19:57:19.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:20.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 492 2024-08-25T19:57:20.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:20.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:57:20.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:20.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:20.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:57:20.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:57:20.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:20.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:20.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:57:20.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:20.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:20.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:57:20.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:57:20.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 495 2024-08-25T19:57:20.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:21.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:21.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:57:21.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:21.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:21.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:57:21.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:57:21.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:21.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:57:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:57:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:57:21.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 494 2024-08-25T19:57:21.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 497 2024-08-25T19:57:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:22.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 496 2024-08-25T19:57:22.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:57:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:57:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:57:22.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 500 2024-08-25T19:57:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:57:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:57:22.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:57:22.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:23.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 498 2024-08-25T19:57:23.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:23.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:57:23.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:23.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:23.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:57:23.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:57:23.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:23.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 501 2024-08-25T19:57:23.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:23.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:57:23.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:23.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:23.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:57:23.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:57:23.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:24.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 499 2024-08-25T19:57:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:24.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 503 2024-08-25T19:57:24.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:24.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:24.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:57:24.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:24.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:24.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:57:24.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:57:24.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:24.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:57:24.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:24.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:24.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:57:24.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:57:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 502 2024-08-25T19:57:25.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:25.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:57:25.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:25.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:25.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:57:25.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:57:25.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:25.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 505 2024-08-25T19:57:25.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:25.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:57:25.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:25.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:25.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:57:25.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:57:25.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:26.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 504 2024-08-25T19:57:26.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:26.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 507 2024-08-25T19:57:26.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:26.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:26.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:57:26.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:26.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:26.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:57:26.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:57:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:57:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:57:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:57:27.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 506 2024-08-25T19:57:27.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:27.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:57:27.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:27.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:27.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:57:27.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:57:27.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:27.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 509 2024-08-25T19:57:27.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:27.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:57:27.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:27.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:27.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:57:27.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:57:27.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:28.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 508 2024-08-25T19:57:28.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:28.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:57:28.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:28.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:28.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:57:28.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:57:28.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:28.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 511 2024-08-25T19:57:28.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:28.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:57:28.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:28.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:28.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:57:28.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:57:28.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:57:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:57:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:57:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:57:29.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:29.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:29.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:57:29.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:57:29.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 510 2024-08-25T19:57:29.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:29.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 513 2024-08-25T19:57:29.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:30.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 512 2024-08-25T19:57:30.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:30.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:57:30.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:30.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:30.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:57:30.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:57:30.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:57:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:57:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:57:30.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 515 2024-08-25T19:57:30.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:31.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 514 2024-08-25T19:57:31.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:31.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 517 2024-08-25T19:57:31.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:31.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:31.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:57:31.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:31.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:31.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:57:31.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:57:31.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:31.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:57:31.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:31.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:31.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:57:31.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:57:32.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 516 2024-08-25T19:57:32.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:32.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:57:32.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:32.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:32.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:57:32.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:57:32.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:32.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 519 2024-08-25T19:57:32.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:32.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:32.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:57:32.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:32.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:32.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:57:32.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:57:33.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 518 2024-08-25T19:57:33.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:33.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:57:33.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:33.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:33.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:57:33.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:57:33.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:33.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 521 2024-08-25T19:57:33.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:33.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:57:33.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:33.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:33.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:57:33.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:57:33.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:57:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:57:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:57:34.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 520 2024-08-25T19:57:34.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:34.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:57:34.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:34.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:34.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:57:34.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:57:34.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:34.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 523 2024-08-25T19:57:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:35.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 522 2024-08-25T19:57:35.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:35.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:57:35.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:35.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:35.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:57:35.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:57:35.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:35.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 525 2024-08-25T19:57:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:57:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:57:35.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:57:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:36.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 524 2024-08-25T19:57:36.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:36.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 527 2024-08-25T19:57:36.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:36.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:36.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:57:36.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:36.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:36.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:57:36.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:57:36.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:36.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:57:36.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:36.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:36.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:57:36.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:57:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 526 2024-08-25T19:57:37.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:37.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 529 2024-08-25T19:57:37.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:37.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:37.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:57:37.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:37.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:37.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:57:37.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:57:37.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:37.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:57:37.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:37.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:37.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:57:37.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:57:38.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 528 2024-08-25T19:57:38.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:38.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 531 2024-08-25T19:57:38.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:38.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:38.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:57:38.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:38.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:57:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:57:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:57:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:57:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:57:39.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 530 2024-08-25T19:57:39.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:39.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:57:39.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:39.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:39.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:57:39.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:57:39.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:39.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:39.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:57:39.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:39.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:39.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:57:39.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:57:39.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 533 2024-08-25T19:57:39.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:40.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 532 2024-08-25T19:57:40.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:40.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:57:40.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:40.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:40.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:57:40.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:57:40.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:40.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:40.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:57:40.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:40.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:40.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:57:40.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:57:40.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 535 2024-08-25T19:57:40.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 534 2024-08-25T19:57:41.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:41.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:41.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:57:41.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:41.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:41.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:57:41.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:57:41.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 537 2024-08-25T19:57:41.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:57:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:57:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:57:42.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 536 2024-08-25T19:57:42.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:42.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:57:42.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:42.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:42.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:57:42.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:57:42.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 539 2024-08-25T19:57:42.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:42.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:57:42.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:42.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:42.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:57:42.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:57:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:43.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 538 2024-08-25T19:57:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:57:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:57:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:57:43.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:43.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 541 2024-08-25T19:57:43.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:43.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:57:43.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:43.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:43.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:57:43.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:57:43.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:44.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 540 2024-08-25T19:57:44.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:44.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:57:44.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:44.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:44.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:57:44.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:57:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:57:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:57:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:57:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 543 2024-08-25T19:57:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:45.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 542 2024-08-25T19:57:45.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:45.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:57:45.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:45.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:45.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:57:45.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:57:45.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:45.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 545 2024-08-25T19:57:45.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:45.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:57:45.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:45.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:45.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:57:45.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:57:45.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:46.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 544 2024-08-25T19:57:46.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:46.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 547 2024-08-25T19:57:46.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:46.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:57:46.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:46.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:46.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:57:46.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:57:46.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:46.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:46.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:57:46.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:46.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:46.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:57:46.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:57:47.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 546 2024-08-25T19:57:47.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:47.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:57:47.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:47.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:47.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:57:47.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:57:47.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:47.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 549 2024-08-25T19:57:47.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:47.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:47.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:57:47.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:47.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:47.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:57:47.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:57:48.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 548 2024-08-25T19:57:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:57:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:57:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:57:48.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:48.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:57:48.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:48.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:48.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:57:48.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:57:48.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 551 2024-08-25T19:57:48.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:49.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:49.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:57:49.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:49.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:49.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:57:49.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:57:49.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 550 2024-08-25T19:57:49.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:49.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:57:49.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:49.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:49.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:57:49.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:57:49.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:49.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 553 2024-08-25T19:57:49.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:50.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:50.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:57:50.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:50.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:50.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:57:50.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:57:50.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:50.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:57:50.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:50.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:50.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:57:50.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:57:50.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 552 2024-08-25T19:57:50.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:50.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 555 2024-08-25T19:57:50.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:51.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 554 2024-08-25T19:57:51.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:51.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 558 2024-08-25T19:57:51.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:57:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:57:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:57:51.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:51.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:57:51.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:51.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:51.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:57:51.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:57:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 556 2024-08-25T19:57:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:52.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 559 2024-08-25T19:57:52.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:52.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:57:52.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:52.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:52.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:57:52.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:57:52.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:52.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:52.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:57:52.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:52.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:52.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:57:52.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:57:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 557 2024-08-25T19:57:53.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:53.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 561 2024-08-25T19:57:53.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:53.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:53.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:57:53.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:53.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:53.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:57:53.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:57:53.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:53.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:57:53.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:53.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:53.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:57:53.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:57:54.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 560 2024-08-25T19:57:54.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:54.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:57:54.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:54.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:54.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:57:54.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:57:54.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:54.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 563 2024-08-25T19:57:54.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:54.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:57:54.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:54.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:54.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:57:54.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:57:54.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:55.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 562 2024-08-25T19:57:55.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:55.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:57:55.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:55.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:55.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:57:55.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:57:55.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:55.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 565 2024-08-25T19:57:55.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:55.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:55.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:57:55.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:55.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:55.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:57:55.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:57:56.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 564 2024-08-25T19:57:56.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:57:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:57:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:57:56.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 567 2024-08-25T19:57:56.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:56.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:56.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:57:56.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:56.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:56.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:57:56.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:57:57.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 566 2024-08-25T19:57:57.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:57.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:57:57.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:57.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:57.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:57:57.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:57:57.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:57.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 569 2024-08-25T19:57:57.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:57.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:57:57.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:57.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:57.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:57:57.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:57:57.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 568 2024-08-25T19:57:58.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:58.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:57:58.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:58.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:58.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:57:58.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:57:58.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:58.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 571 2024-08-25T19:57:58.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:58.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:57:58.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:58.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:58.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:57:58.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:57:58.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:57:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:57:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:57:59.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 570 2024-08-25T19:57:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:57:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:57:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:57:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:57:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:57:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:57:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:57:59.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 573 2024-08-25T19:57:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:00.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 572 2024-08-25T19:58:00.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:58:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:58:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:58:00.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:00.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:58:00.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:00.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:00.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:58:00.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:58:00.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 575 2024-08-25T19:58:00.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:01.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 574 2024-08-25T19:58:01.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:01.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 577 2024-08-25T19:58:01.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:01.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:58:01.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:01.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:01.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:58:01.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:58:01.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:01.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:01.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:58:01.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:01.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:01.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:58:01.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:58:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 576 2024-08-25T19:58:02.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:02.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:58:02.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:02.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:02.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:58:02.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:58:02.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:02.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 579 2024-08-25T19:58:02.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:02.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:58:02.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:02.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:02.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:58:02.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:58:02.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:03.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 578 2024-08-25T19:58:03.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:03.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:58:03.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:03.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:03.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:58:03.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:58:03.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:03.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 581 2024-08-25T19:58:03.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:03.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:58:03.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:03.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:03.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:58:03.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:58:03.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:04.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:04.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:58:04.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:04.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:58:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:58:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 580 2024-08-25T19:58:04.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:04.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:58:04.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:04.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:04.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:58:04.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:58:04.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 583 2024-08-25T19:58:04.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:05.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 582 2024-08-25T19:58:05.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:05.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:58:05.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:05.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:05.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:58:05.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:58:05.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:05.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:05.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:58:05.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:05.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:05.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:58:05.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:58:05.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 585 2024-08-25T19:58:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:06.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 584 2024-08-25T19:58:06.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:06.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 587 2024-08-25T19:58:06.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:06.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:06.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:58:06.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:06.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:06.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:58:06.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:58:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:58:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:58:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:58:07.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 586 2024-08-25T19:58:07.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:07.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:07.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:58:07.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:07.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:07.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:58:07.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:58:07.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 589 2024-08-25T19:58:07.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:07.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:58:07.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:07.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:07.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:58:07.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:58:07.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:08.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 588 2024-08-25T19:58:08.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:08.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:58:08.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:08.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:08.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:58:08.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:58:08.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:08.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 591 2024-08-25T19:58:08.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:08.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:08.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:58:08.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:08.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:08.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:58:08.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:58:09.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:09.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:58:09.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:09.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:09.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:58:09.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:58:09.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 590 2024-08-25T19:58:09.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:09.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:58:09.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:09.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:09.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:58:09.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:58:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:09.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 593 2024-08-25T19:58:09.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 592 2024-08-25T19:58:10.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:58:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:58:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:58:10.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:10.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 595 2024-08-25T19:58:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:58:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:58:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:58:10.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 594 2024-08-25T19:58:11.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 597 2024-08-25T19:58:11.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:11.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:11.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:58:11.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:11.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:11.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:58:11.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:58:11.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:11.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:58:11.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:11.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:11.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:58:11.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:58:12.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 596 2024-08-25T19:58:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:12.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 599 2024-08-25T19:58:12.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:12.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:12.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:58:12.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:12.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:12.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:58:12.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:58:12.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:12.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:58:12.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:12.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:12.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:58:12.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:58:13.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 598 2024-08-25T19:58:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:13.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 601 2024-08-25T19:58:13.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:58:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:58:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:58:13.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:13.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:58:13.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:13.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:13.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:58:13.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:58:14.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 600 2024-08-25T19:58:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:58:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:58:14.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:58:14.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:14.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 603 2024-08-25T19:58:14.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:58:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:58:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:58:14.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:15.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 602 2024-08-25T19:58:15.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:15.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:58:15.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:15.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:15.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:58:15.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:58:15.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:15.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:15.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:58:15.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:15.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:15.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:58:15.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:58:15.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 605 2024-08-25T19:58:15.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 604 2024-08-25T19:58:16.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:16.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:58:16.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:16.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:16.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:58:16.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:58:16.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:16.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 607 2024-08-25T19:58:16.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:16.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:58:16.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:16.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:16.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:58:16.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:58:16.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:17.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 606 2024-08-25T19:58:17.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:17.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:58:17.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:17.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:17.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:58:17.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:58:17.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:17.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:17.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:58:17.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:17.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:17.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:58:17.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:58:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 609 2024-08-25T19:58:17.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 608 2024-08-25T19:58:18.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:18.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:58:18.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:18.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:18.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:58:18.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:58:18.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:18.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 611 2024-08-25T19:58:18.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:18.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:58:18.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:18.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:18.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:58:18.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:58:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:19.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 610 2024-08-25T19:58:19.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:19.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:58:19.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:19.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:19.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:58:19.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:58:19.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:19.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:19.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:58:19.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:19.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:19.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:58:19.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:58:19.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 613 2024-08-25T19:58:19.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:58:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:58:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:58:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 612 2024-08-25T19:58:20.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:20.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:20.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:58:20.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:20.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:20.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:58:20.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:58:20.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 615 2024-08-25T19:58:20.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:21.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 614 2024-08-25T19:58:21.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:21.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 617 2024-08-25T19:58:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:58:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:58:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:58:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:58:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:21.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:58:21.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:58:22.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 616 2024-08-25T19:58:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:58:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:58:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:58:22.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:58:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:58:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:58:22.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 619 2024-08-25T19:58:22.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:23.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 618 2024-08-25T19:58:23.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:23.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:58:23.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:23.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:23.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:58:23.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:58:23.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:23.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 621 2024-08-25T19:58:23.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:23.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:23.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:58:23.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:23.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:23.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:58:23.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:58:24.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:24.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:58:24.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:24.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:24.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:58:24.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:58:24.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 620 2024-08-25T19:58:24.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:24.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:58:24.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:24.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:24.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:58:24.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:58:24.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:24.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 623 2024-08-25T19:58:24.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:25.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 622 2024-08-25T19:58:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:58:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:58:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:58:25.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:25.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:25.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T19:58:25.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:25.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:25.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T19:58:25.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T19:58:25.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 625 2024-08-25T19:58:25.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:26.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 624 2024-08-25T19:58:26.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:26.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 627 2024-08-25T19:58:26.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:58:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:58:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:58:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T19:58:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T19:58:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T19:58:27.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 626 2024-08-25T19:58:27.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:27.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 629 2024-08-25T19:58:27.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:27.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:27.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:58:27.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:27.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:27.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:58:27.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:58:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T19:58:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:27.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T19:58:27.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T19:58:28.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 628 2024-08-25T19:58:28.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:28.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 631 2024-08-25T19:58:28.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:28.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:28.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:58:28.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:28.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:28.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:58:28.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:58:28.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:28.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T19:58:28.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:28.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:28.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T19:58:28.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T19:58:29.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 630 2024-08-25T19:58:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:58:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:58:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:58:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:29.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 633 2024-08-25T19:58:29.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:29.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T19:58:29.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:29.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:29.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T19:58:29.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T19:58:29.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:30.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 632 2024-08-25T19:58:30.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:30.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:58:30.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:30.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:30.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:58:30.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:58:30.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:30.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:30.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T19:58:30.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:30.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:30.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T19:58:30.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T19:58:30.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 635 2024-08-25T19:58:30.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:31.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 634 2024-08-25T19:58:31.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:58:31.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 35385 2024-08-25T19:58:31.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:31.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 637 2024-08-25T19:58:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:31.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:31.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T19:58:31.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:31.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:31.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T19:58:31.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T19:58:32.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 636 2024-08-25T19:58:32.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:58:32.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 35525 2024-08-25T19:58:32.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:32.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:32.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T19:58:32.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:32.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:32.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T19:58:32.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T19:58:33.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 638 2024-08-25T19:58:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:58:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 35658 2024-08-25T19:58:33.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:33.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:33.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T19:58:33.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:33.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:33.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T19:58:33.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T19:58:34.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 639 2024-08-25T19:58:34.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:58:34.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 35810 2024-08-25T19:58:34.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:34.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T19:58:34.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T19:58:34.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T19:58:34.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T19:58:34.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T19:58:34.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T19:58:35.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 640 2024-08-25T19:58:35.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:35.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:58:35.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 35963 2024-08-25T19:58:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 641 2024-08-25T19:58:36.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:36.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:58:36.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 36116 2024-08-25T19:58:37.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 642 2024-08-25T19:58:37.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T19:58:37.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:58:37.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 36269 2024-08-25T19:58:38.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:58:38.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 36442 2024-08-25T19:58:39.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T19:58:39.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 36616 2024-08-25T19:58:40.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T19:58:40.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T19:58:40.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 35252 2024-08-25T20:01:05.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:01:05.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:01:05.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:01:05.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 35386 2024-08-25T20:01:06.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:01:06.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:01:06.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:01:06.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 35526 2024-08-25T20:01:07.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:01:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:01:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:01:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 35659 2024-08-25T20:01:08.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:01:08.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:01:08.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:01:08.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 35811 2024-08-25T20:01:09.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:01:09.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:01:09.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:01:09.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 35964 2024-08-25T20:01:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:01:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:01:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:01:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 36117 2024-08-25T20:01:11.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:01:11.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:01:11.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:01:11.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 36270 2024-08-25T20:01:12.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:01:12.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:01:12.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:01:12.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 36443 2024-08-25T20:01:13.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:01:13.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:01:13.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:01:13.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 36617 2024-08-25T20:01:14.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:01:14.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:01:14.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:01:14.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:01:14.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:01:14.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:01:14.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 1 2024-08-25T20:01:14.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:01:14.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd0 2024-08-25T20:01:14.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary1 2024-08-25T20:01:14.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary1 2024-08-25T20:01:14.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:01:14.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary1 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 2 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd1 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary2 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary2 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary2 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 3 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd2 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary3 2024-08-25T20:01:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary3 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary3 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 4 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd3 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary4 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary4 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary4 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 5 2024-08-25T20:01:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:01:14.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd4 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary5 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary5 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary5 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 6 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd5 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary6 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary6 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary6 2024-08-25T20:01:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:01:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:01:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 7 2024-08-25T20:01:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd6 2024-08-25T20:01:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary7 2024-08-25T20:01:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary7 2024-08-25T20:01:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:01:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary7 2024-08-25T20:01:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:01:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:01:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 8 2024-08-25T20:01:14.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd7 2024-08-25T20:01:14.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 9 2024-08-25T20:01:14.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary8 2024-08-25T20:01:14.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary8 2024-08-25T20:01:14.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:01:14.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:01:14.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary8 2024-08-25T20:01:14.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:01:14.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd8 2024-08-25T20:01:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:01:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary9 2024-08-25T20:01:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary9 2024-08-25T20:01:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:01:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary9 2024-08-25T20:01:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:01:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 10 2024-08-25T20:01:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 42585 2024-08-25T20:01:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd9 2024-08-25T20:01:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary10 2024-08-25T20:01:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary10 2024-08-25T20:01:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:01:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary10 2024-08-25T20:01:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:01:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:01:15.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:01:15.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:01:15.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:01:15.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:01:15.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:01:15.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:01:15.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:01:15.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:01:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:01:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:01:15.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:01:15.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:01:15.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:01:15.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:01:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:01:15.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:01:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:01:15.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:02:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=098d2d1f751acef337ce5410722cc188 2024-08-25T20:02:09.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-08-25T20:02:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary3 2024-08-25T20:02:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:02:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-08-25T20:02:10.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=04c0f94ca37da0ff838737dca8cb4ae1 2024-08-25T20:02:10.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-08-25T20:02:10.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:02:10.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-08-25T20:02:10.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:10.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:10.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:02:10.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:10.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:10.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:10.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:10.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary6 2024-08-25T20:02:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:10.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:02:10.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-08-25T20:02:10.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=0b46a7817364e0d1d142a7c1974b3310 2024-08-25T20:02:10.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-08-25T20:02:10.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=273c44035fb892d1ac290c58192f4f8f 2024-08-25T20:02:10.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-08-25T20:02:10.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary8 2024-08-25T20:02:10.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:10.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:10.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:02:10.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-08-25T20:02:11.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary2 2024-08-25T20:02:11.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:11.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:11.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:02:11.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-08-25T20:02:11.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:02:11.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:02:11.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:02:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-08-25T20:02:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:02:11.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:11.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:11.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:11.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:11.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-08-25T20:02:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:02:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:11.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-08-25T20:02:11.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:11.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:11.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:02:11.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:11.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:11.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:11.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:11.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:02:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:02:11.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:11.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:11.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:11.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:11.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:11.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:02:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:02:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6a6a0d98-0015-4435-a50c-afd86813e19e 2024-08-25T20:02:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:11.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:11.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:02:11.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=9b09423049a10597d61bd679ae821407 2024-08-25T20:02:11.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-08-25T20:02:11.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=52540cb803b22fe1b11f84adeb12870d 2024-08-25T20:02:11.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-08-25T20:02:11.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary10 2024-08-25T20:02:11.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:11.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:11.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:02:11.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-08-25T20:02:11.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary7 2024-08-25T20:02:11.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:11.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:11.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:02:11.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-08-25T20:02:12.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=7679df7d1cec53386b4979abc5de6caf 2024-08-25T20:02:12.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-08-25T20:02:12.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:02:12.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:02:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:02:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:12.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:12.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:02:12.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:02:12.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:12.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:12.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:02:12.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:12.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:12.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:12.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:12.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:02:12.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:02:12.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:12.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:12.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:02:12.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:12.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:02:12.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:12.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:12.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:12.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:12.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:12.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:02:12.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-08-25T20:02:12.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:12.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:12.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:02:12.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:12.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:12.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:12.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:12.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:12.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-08-25T20:02:12.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:12.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:12.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:02:12.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:12.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:02:12.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4b479bdf-9f6e-4198-8e79-ff93a2d21c04 2024-08-25T20:02:12.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:12.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:12.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:12.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:12.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:02:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6836838d-c369-4977-8900-df8dc7a72cf6 2024-08-25T20:02:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:12.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:12.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:12.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:02:12.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:12.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:12.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:02:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary5 2024-08-25T20:02:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:02:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-08-25T20:02:12.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:02:12.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ba93f24c-de3f-4b9f-9f95-a8a9c5aa159d 2024-08-25T20:02:12.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:12.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:12.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:12.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:12.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:12.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:02:13.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=b1a255798bf5daa1b5a547f0b914d314 2024-08-25T20:02:13.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-08-25T20:02:13.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:02:13.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-08-25T20:02:13.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:13.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:13.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:02:13.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:13.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:13.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:13.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:13.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:13.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:02:13.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:13.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:13.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:02:13.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:13.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:13.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:13.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:13.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:13.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:02:13.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:02:13.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:13.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:13.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:02:13.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:13.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:13.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:13.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:13.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:13.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:02:13.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary1 2024-08-25T20:02:13.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:13.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:13.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:02:13.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-08-25T20:02:13.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:02:13.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:13.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:13.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:02:13.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:13.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:13.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:13.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:02:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f6047f7-e198-4c21-aeb1-fcc11903a8a7 2024-08-25T20:02:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:02:13.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:13.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:13.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:02:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:02:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 12419548-8f80-4684-838b-1b7f43caadcb 2024-08-25T20:02:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:13.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:13.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:13.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:02:13.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:02:13.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6a6a0d98-0015-4435-a50c-afd86813e19e 2024-08-25T20:02:13.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:13.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:13.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:13.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:13.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:13.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:13.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:13.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:02:14.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:02:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-08-25T20:02:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:02:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:14.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:14.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:14.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:14.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:14.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:02:14.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:14.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:14.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:02:14.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:14.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:14.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:14.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:02:14.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:02:14.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d07b684d-ae92-495b-af63-7a0de0e70d08 2024-08-25T20:02:14.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:14.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:14.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:14.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:14.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:14.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:14.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:14.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:02:14.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=2316985fc4aba593b951fff37b55dee5 2024-08-25T20:02:14.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-08-25T20:02:14.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=4207147345a4c37a77fb878ff2c0079f 2024-08-25T20:02:14.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-08-25T20:02:14.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary4 2024-08-25T20:02:14.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:14.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:14.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:02:14.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-08-25T20:02:14.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:02:14.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:14.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:14.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:02:14.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:14.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:14.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:14.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:14.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:14.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:02:14.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:02:14.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:14.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:14.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:02:14.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:14.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:14.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:14.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:14.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:14.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:02:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:02:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:02:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:02:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary9 2024-08-25T20:02:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:02:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-08-25T20:02:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:02:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6836838d-c369-4977-8900-df8dc7a72cf6 2024-08-25T20:02:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:14.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:14.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:14.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:14.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:02:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:02:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ba93f24c-de3f-4b9f-9f95-a8a9c5aa159d 2024-08-25T20:02:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:02:14.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4b479bdf-9f6e-4198-8e79-ff93a2d21c04 2024-08-25T20:02:14.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:14.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:14.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:14.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:14.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:14.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:14.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:14.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:02:14.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:14.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:14.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:02:15.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:02:15.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:02:15.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:02:15.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:15.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:15.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:02:15.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:15.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:15.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:15.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:15.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:15.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:02:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-08-25T20:02:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:02:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-08-25T20:02:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:02:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:15.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:02:15.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4c2ba3a4-f815-4f35-86fb-0f609016cfb2 2024-08-25T20:02:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:15.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:15.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:15.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:02:15.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:02:15.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:15.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:15.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:02:15.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:15.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:15.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:15.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:15.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:15.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:02:15.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:02:15.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:15.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:15.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:02:15.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:15.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:15.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:15.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:15.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:15.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:02:15.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:02:15.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f6047f7-e198-4c21-aeb1-fcc11903a8a7 2024-08-25T20:02:15.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:15.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:15.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:15.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:15.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:15.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:15.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:15.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:02:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:02:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 12419548-8f80-4684-838b-1b7f43caadcb 2024-08-25T20:02:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:15.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:15.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:15.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:02:16.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:02:16.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:02:16.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:16.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:16.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:16.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:16.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:02:16.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:02:16.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:02:16.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:16.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:02:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:02:16.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5f5a643b-a56b-4251-aa94-d881dc6cc262 2024-08-25T20:02:16.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:16.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:16.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:16.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:16.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:16.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:02:16.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 44ece9fc-ce13-4f35-b342-6d5322202441 2024-08-25T20:02:16.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:16.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:16.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:16.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:16.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:16.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:16.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:16.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:02:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:02:16.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:02:16.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:16.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:16.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:02:16.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:16.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:16.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:16.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:16.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:16.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:02:16.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:02:16.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d07b684d-ae92-495b-af63-7a0de0e70d08 2024-08-25T20:02:16.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:16.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:16.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:16.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:16.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:02:17.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:02:17.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:17.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:17.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:02:17.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:17.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:17.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:17.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:17.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:17.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:02:17.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:02:17.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4c2ba3a4-f815-4f35-86fb-0f609016cfb2 2024-08-25T20:02:17.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:17.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:17.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:17.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:17.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:17.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:17.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:17.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:02:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:02:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:02:17.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:17.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:17.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:17.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:17.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:17.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:02:17.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:02:17.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6a6a0d98-0015-4435-a50c-afd86813e19e 2024-08-25T20:02:17.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:17.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:17.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:17.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:17.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:17.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:17.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:17.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:02:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:02:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:02:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:02:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:02:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:18.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:02:18.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:02:18.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5f5a643b-a56b-4251-aa94-d881dc6cc262 2024-08-25T20:02:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:02:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 44ece9fc-ce13-4f35-b342-6d5322202441 2024-08-25T20:02:18.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:18.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:18.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:18.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:18.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:18.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:18.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:18.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:02:18.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:18.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:18.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:02:18.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:02:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:02:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:18.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:18.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:02:18.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:02:18.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:18.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:18.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:02:18.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:18.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:18.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:18.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:18.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:02:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:02:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:18.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:02:18.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:18.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:02:18.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:02:18.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6836838d-c369-4977-8900-df8dc7a72cf6 2024-08-25T20:02:18.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:18.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:18.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:18.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:18.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:18.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:18.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:18.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:18.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:02:18.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ba93f24c-de3f-4b9f-9f95-a8a9c5aa159d 2024-08-25T20:02:18.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:18.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:18.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:18.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:18.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:02:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4b479bdf-9f6e-4198-8e79-ff93a2d21c04 2024-08-25T20:02:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:18.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:18.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:18.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:18.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:18.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:18.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:19.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:02:19.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:19.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:19.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:02:19.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:19.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:19.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:19.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:19.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:19.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:02:19.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:02:19.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:19.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:19.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:02:19.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:19.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:19.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:19.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:19.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:19.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:02:19.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:02:19.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f6047f7-e198-4c21-aeb1-fcc11903a8a7 2024-08-25T20:02:19.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:19.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:19.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:19.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:19.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:19.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:02:19.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 12419548-8f80-4684-838b-1b7f43caadcb 2024-08-25T20:02:19.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:19.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:19.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:19.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:19.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:19.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:19.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:19.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:20.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:02:20.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:20.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:20.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:02:20.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:20.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:20.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:20.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:20.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:20.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:02:20.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:02:20.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d07b684d-ae92-495b-af63-7a0de0e70d08 2024-08-25T20:02:20.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:20.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:20.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:20.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:20.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:21.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:02:21.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:21.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:21.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:02:21.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:21.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:21.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:21.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:21.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:21.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:02:21.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:02:21.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4c2ba3a4-f815-4f35-86fb-0f609016cfb2 2024-08-25T20:02:21.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:21.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:21.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:21.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:22.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:02:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:02:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:02:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:02:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:02:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:02:22.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:02:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5f5a643b-a56b-4251-aa94-d881dc6cc262 2024-08-25T20:02:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:02:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 44ece9fc-ce13-4f35-b342-6d5322202441 2024-08-25T20:02:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:22.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:22.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:22.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:22.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:22.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:22.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:25.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:02:25.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:25.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:25.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:02:25.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:25.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:25.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:25.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:25.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:25.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:02:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:02:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6a6a0d98-0015-4435-a50c-afd86813e19e 2024-08-25T20:02:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:25.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:25.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:25.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:26.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:02:26.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:26.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:26.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:02:26.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:26.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:26.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:26.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:26.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:26.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:02:26.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:02:26.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:26.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:26.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:02:26.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:26.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:26.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:26.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:26.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:02:26.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:26.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:02:26.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:26.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:26.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:02:26.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:26.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:26.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:26.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:26.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:26.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:02:26.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:02:26.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6836838d-c369-4977-8900-df8dc7a72cf6 2024-08-25T20:02:26.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:26.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:26.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:26.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:26.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:26.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:26.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:26.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:02:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ba93f24c-de3f-4b9f-9f95-a8a9c5aa159d 2024-08-25T20:02:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:02:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4b479bdf-9f6e-4198-8e79-ff93a2d21c04 2024-08-25T20:02:26.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:26.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:26.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:26.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:26.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:26.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:26.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:26.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:26.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:26.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:26.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:26.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:26.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:27.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:02:27.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:27.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:27.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:02:27.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:27.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:27.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:27.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:27.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:27.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:02:27.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:02:27.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:27.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:27.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:02:27.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:27.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:27.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:27.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:27.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:27.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:02:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:02:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f6047f7-e198-4c21-aeb1-fcc11903a8a7 2024-08-25T20:02:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:27.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:27.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:27.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:27.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:27.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:27.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:27.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:02:27.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 12419548-8f80-4684-838b-1b7f43caadcb 2024-08-25T20:02:27.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:02:27.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:02:27.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:27.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:01:58' 2024-08-25T20:02:27.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:27.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:02:27.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:27.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:02:28.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:02:28.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:28.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:28.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:02:28.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:28.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:28.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:28.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:28.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:28.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:02:28.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:02:28.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d07b684d-ae92-495b-af63-7a0de0e70d08 2024-08-25T20:02:28.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:28.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:28.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:28.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:28.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:28.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:28.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:28.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:28.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:28.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:28.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:28.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:28.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:28.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:28.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:02:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:28.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:28.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:28.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:02:28.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:28.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:02:28.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:28.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:28.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:28.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:28.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:28.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:28.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-08-25T20:02:28.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:28.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:28.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:02:28.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:28.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:28.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:28.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:28.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:29.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:02:29.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:29.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:29.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:02:29.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:29.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:29.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:29.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:29.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:29.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:02:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:02:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4c2ba3a4-f815-4f35-86fb-0f609016cfb2 2024-08-25T20:02:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:29.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:29.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:29.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:29.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:29.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:29.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:29.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:29.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:29.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:29.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:29.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:29.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:29.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:29.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:29.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:29.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:02:29.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:29.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:29.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:29.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:29.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:29.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:29.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:29.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:02:29.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:29.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:02:29.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:29.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:29.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:29.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:29.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:29.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:29.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-08-25T20:02:29.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:29.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:29.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:02:29.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:29.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:29.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:29.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:29.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-08-25T20:02:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:02:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-08-25T20:02:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:02:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d07b684d-ae92-495b-af63-7a0de0e70d08 2024-08-25T20:02:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:29.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:29.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:29.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:29.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:29.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:29.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:29.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:29.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:29.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:29.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:29.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:02:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:29.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:29.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:30.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:30.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:30.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:02:30.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:30.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:02:30.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:30.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:30.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:30.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary5 2024-08-25T20:02:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:02:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:02:30.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-08-25T20:02:30.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:02:30.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:30.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:30.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:02:30.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:30.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:30.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:30.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:30.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:30.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:02:30.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:02:30.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:30.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:30.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:02:30.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:30.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:30.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:30.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:30.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:30.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:02:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:02:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5f5a643b-a56b-4251-aa94-d881dc6cc262 2024-08-25T20:02:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:02:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 44ece9fc-ce13-4f35-b342-6d5322202441 2024-08-25T20:02:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:30.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:30.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:30.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:30.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:30.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:30.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:30.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:30.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:30.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:30.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:30.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:30.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:30.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:30.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:30.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:30.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:30.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:30.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:30.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:30.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:30.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:30.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:30.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:30.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:30.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:30.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:02:30.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:30.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:30.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:30.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:30.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:30.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:30.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:30.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:30.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:30.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:02:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-08-25T20:02:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:30.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:30.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:02:30.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:30.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:30.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:30.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:30.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:30.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-08-25T20:02:30.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:30.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:30.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:02:30.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:30.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:30.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:02:30.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:30.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:02:30.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:30.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:02:30.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:30.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:30.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-08-25T20:02:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:02:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-08-25T20:02:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:30.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:30.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:02:30.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:30.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:30.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:30.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:30.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:02:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4c2ba3a4-f815-4f35-86fb-0f609016cfb2 2024-08-25T20:02:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:30.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:30.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:30.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:30.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:30.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:30.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:30.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:02:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:30.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:30.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:31.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:31.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:31.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:02:31.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:31.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:02:31.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:31.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:31.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:31.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:31.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:31.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:31.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary1 2024-08-25T20:02:31.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:31.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:31.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:02:31.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:02:31.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-08-25T20:02:31.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:02:31.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:02:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:02:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:02:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:02:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:02:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-08-25T20:02:31.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-08-25T20:02:31.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd0 2024-08-25T20:02:31.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd1 2024-08-25T20:02:31.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:02:31.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:02:31.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:02:31.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:02:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-08-25T20:02:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:02:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:31.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-08-25T20:02:31.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:31.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:31.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:02:31.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:31.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:31.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:31.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:31.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:31.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-08-25T20:02:31.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:31.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-08-25T20:02:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:02:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5f5a643b-a56b-4251-aa94-d881dc6cc262 2024-08-25T20:02:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:31.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:02:31.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 44ece9fc-ce13-4f35-b342-6d5322202441 2024-08-25T20:02:31.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:31.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:31.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:31.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:31.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:31.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:31.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:31.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:31.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:31.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:31.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:31.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:31.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:31.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:31.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:31.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:31.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:31.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:31.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:31.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:31.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:31.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:31.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:31.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:31.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:31.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:31.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:31.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:31.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:31.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:31.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:31.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:02:31.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:31.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:31.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:31.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:31.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:31.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:31.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:31.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:31.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:31.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:02:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:32.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:32.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:32.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:02:32.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:32.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:32.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:02:32.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:32.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:02:32.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:32.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:02:32.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:32.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:32.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:32.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:32.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:32.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:32.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:32.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary4 2024-08-25T20:02:32.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:32.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:32.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:02:32.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:02:32.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-08-25T20:02:32.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:32.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:32.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:32.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary9 2024-08-25T20:02:32.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:32.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:32.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:02:32.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:02:32.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-08-25T20:02:33.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:02:33.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:02:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:02:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:02:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-08-25T20:02:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:02:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:02:33.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-08-25T20:02:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd2 2024-08-25T20:02:33.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:02:33.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:02:33.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd3 2024-08-25T20:02:33.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:02:33.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:02:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:02:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:02:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:33.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:33.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:02:33.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:02:33.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6a6a0d98-0015-4435-a50c-afd86813e19e 2024-08-25T20:02:33.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:33.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:33.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:33.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:33.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:33.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:33.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:33.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:33.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:33.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:33.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:33.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:33.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:33.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:33.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:02:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:34.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:34.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:34.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:02:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:02:34.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:34.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:34.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:34.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-08-25T20:02:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:02:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:34.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:02:34.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:34.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:34.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:02:34.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:34.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:34.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:34.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:34.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:34.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:02:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:02:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:02:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:02:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:34.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:34.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:34.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:34.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:02:34.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:34.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:34.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:34.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:34.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:34.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:02:34.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:34.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:02:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:02:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ba93f24c-de3f-4b9f-9f95-a8a9c5aa159d 2024-08-25T20:02:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:02:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4b479bdf-9f6e-4198-8e79-ff93a2d21c04 2024-08-25T20:02:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:02:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6836838d-c369-4977-8900-df8dc7a72cf6 2024-08-25T20:02:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:34.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:34.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:34.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:34.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:34.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:34.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:34.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:34.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:34.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:34.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:34.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:34.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:34.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:34.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:34.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:34.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:34.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:34.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:34.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:34.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:34.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:34.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:34.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:34.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:34.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:34.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:34.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:02:34.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:34.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:34.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:34.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:34.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-08-25T20:02:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:02:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:35.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:35.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-08-25T20:02:35.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:35.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:35.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:02:35.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:35.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:02:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:35.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:35.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:35.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:35.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:35.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:35.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:35.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:35.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:02:35.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:02:35.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:35.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:35.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-08-25T20:02:35.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:35.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:35.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:02:35.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:35.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:35.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:35.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:35.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:35.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:35.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:35.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:02:35.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:02:35.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:02:35.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6a6a0d98-0015-4435-a50c-afd86813e19e 2024-08-25T20:02:35.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:35.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:35.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:35.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:35.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:35.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:35.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:35.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:35.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:35.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-08-25T20:02:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:02:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-08-25T20:02:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:02:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:35.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:35.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:35.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:35.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:35.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:35.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:35.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:35.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:35.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:35.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:35.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:35.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:02:35.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:35.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:35.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:35.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:35.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:35.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:35.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:35.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:02:35.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:02:35.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:35.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:35.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:35.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:35.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:35.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:35.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary3 2024-08-25T20:02:35.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:35.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:35.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:02:35.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:02:35.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-08-25T20:02:35.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:02:35.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:35.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:35.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:02:35.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:35.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:35.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:35.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:35.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:35.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:02:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:02:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:02:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:35.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:35.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:02:35.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:02:35.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f6047f7-e198-4c21-aeb1-fcc11903a8a7 2024-08-25T20:02:35.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:35.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:35.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:35.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:35.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:35.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:35.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:35.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:35.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:35.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:35.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:35.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:35.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:35.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:35.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:35.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:35.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:35.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:35.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:35.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:35.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:02:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:02:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 12419548-8f80-4684-838b-1b7f43caadcb 2024-08-25T20:02:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:35.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:35.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:35.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:35.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:35.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:35.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:35.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:35.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:35.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:35.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:35.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:35.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:35.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:35.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:35.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:35.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:02:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:02:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:35.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:35.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:36.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:36.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:36.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:02:36.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:36.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:02:36.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:36.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:36.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-08-25T20:02:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:02:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:36.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:36.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:36.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:02:36.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:02:36.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:36.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:36.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:36.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:36.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:36.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:36.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-08-25T20:02:36.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:36.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:36.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:02:36.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:36.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:36.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:36.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:02:36.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:02:36.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-08-25T20:02:36.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:02:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:36.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:36.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-08-25T20:02:36.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-08-25T20:02:36.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:36.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:36.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:36.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:36.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-08-25T20:02:36.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:36.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-08-25T20:02:36.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:02:36.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:02:36.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6836838d-c369-4977-8900-df8dc7a72cf6 2024-08-25T20:02:36.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:36.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:36.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:36.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:36.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:36.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:02:36.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:02:36.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:36.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:36.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-08-25T20:02:36.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:36.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:36.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:36.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:36.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:36.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:02:36.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ba93f24c-de3f-4b9f-9f95-a8a9c5aa159d 2024-08-25T20:02:36.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:36.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:36.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:36.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:36.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:02:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4b479bdf-9f6e-4198-8e79-ff93a2d21c04 2024-08-25T20:02:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:36.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:36.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:36.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:36.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:02:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:36.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:36.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:36.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:36.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:36.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:36.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:36.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:36.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:36.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:36.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:36.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:36.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:36.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:36.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:36.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:36.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:36.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:36.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:36.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:36.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:02:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:02:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:36.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:36.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:36.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd4 2024-08-25T20:02:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:02:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:02:36.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:36.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:36.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:02:36.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:36.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:02:36.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:36.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:36.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:02:36.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:36.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:36.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:02:36.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:36.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:36.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:36.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:36.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:36.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:02:36.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:02:36.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:36.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:36.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:36.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:36.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:36.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:36.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:36.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary8 2024-08-25T20:02:36.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:36.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:36.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:36.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:02:36.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:02:36.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-08-25T20:02:36.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:36.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:36.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:36.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:36.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary2 2024-08-25T20:02:36.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:36.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:36.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:02:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:02:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-08-25T20:02:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary6 2024-08-25T20:02:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:02:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:02:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-08-25T20:02:37.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-08-25T20:02:37.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:37.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:37.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:02:37.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:37.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:37.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:37.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:37.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:37.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-08-25T20:02:37.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-08-25T20:02:37.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:37.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:37.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:02:37.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:37.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:02:37.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:02:37.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:37.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:37.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-08-25T20:02:37.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:02:37.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f6047f7-e198-4c21-aeb1-fcc11903a8a7 2024-08-25T20:02:37.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:37.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:37.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:37.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:37.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:37.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:37.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:37.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:37.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:37.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:37.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:37.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:37.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:37.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:37.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:37.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:37.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:37.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:37.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:37.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:37.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:02:37.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:37.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:37.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:37.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:37.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:37.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:02:37.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 12419548-8f80-4684-838b-1b7f43caadcb 2024-08-25T20:02:37.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:02:37.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:02:37.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:02:37.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:02:28' 2024-08-25T20:02:37.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:02:37.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:02:37.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:02:37.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:02:37.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:02:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:02:37.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:02:37.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:02:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:02:37.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:02:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:37.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-08-25 20:02:28' 2024-08-25T20:02:37.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:37.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:37.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:02:37.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:02:37.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:02:28' 2024-08-25T20:02:37.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:02:37.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:02:37.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:02:37.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:02:37.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:02:37.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:02:37.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:02:37.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:02:37.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-08-25T20:02:37.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:02:37.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:02:37.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-08-25T20:02:37.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:02:37.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:02:37.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-08-25T20:02:37.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:37.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:02:37.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:37.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:02:37.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:37.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:37.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:37.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:37.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:37.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:37.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary10 2024-08-25T20:02:37.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:37.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:37.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:02:37.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:02:37.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-08-25T20:02:37.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:02:28' 2024-08-25T20:02:37.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:02:37.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:02:37.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:02:28' 2024-08-25T20:02:37.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:02:37.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:02:37.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:02:37.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:02:37.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:02:37.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:37.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:02:37.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary7 2024-08-25T20:02:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:02:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:02:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:02:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:02:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-08-25T20:02:37.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd5 2024-08-25T20:02:37.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:02:37.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:02:37.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd7 2024-08-25T20:02:37.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:02:37.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:02:37.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd6 2024-08-25T20:02:37.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:02:37.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:02:38.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:02:38.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:02:38.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:02:38.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:02:38.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:02:38.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:02:38.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-08-25T20:02:38.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-08-25T20:02:38.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd8 2024-08-25T20:02:38.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:02:38.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:02:38.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd9 2024-08-25T20:02:38.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:02:38.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:03:22.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=b1a255798bf5daa1b5a547f0b914d314 2024-08-25T20:03:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-08-25T20:03:23.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ b1a255798bf5daa1b5a547f0b914d314 != \b\1\a\2\5\5\7\9\8\b\f\5\d\a\a\1\b\5\a\5\4\7\f\0\b\9\1\4\d\3\1\4 ]] 2024-08-25T20:03:23.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:03:23.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 42586 2024-08-25T20:03:24.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=7679df7d1cec53386b4979abc5de6caf 2024-08-25T20:03:24.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-08-25T20:03:25.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 7679df7d1cec53386b4979abc5de6caf != \7\6\7\9\d\f\7\d\1\c\e\c\5\3\3\8\6\b\4\9\7\9\a\b\c\5\d\e\6\c\a\f ]] 2024-08-25T20:03:27.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=4207147345a4c37a77fb878ff2c0079f 2024-08-25T20:03:27.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-08-25T20:03:27.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 4207147345a4c37a77fb878ff2c0079f != \4\2\0\7\1\4\7\3\4\5\a\4\c\3\7\a\7\7\f\b\8\7\8\f\f\2\c\0\0\7\9\f ]] 2024-08-25T20:03:27.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=2316985fc4aba593b951fff37b55dee5 2024-08-25T20:03:27.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-08-25T20:03:28.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 2316985fc4aba593b951fff37b55dee5 != \2\3\1\6\9\8\5\f\c\4\a\b\a\5\9\3\b\9\5\1\f\f\f\3\7\b\5\5\d\e\e\5 ]] 2024-08-25T20:03:28.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=273c44035fb892d1ac290c58192f4f8f 2024-08-25T20:03:28.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-08-25T20:03:28.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 273c44035fb892d1ac290c58192f4f8f != \2\7\3\c\4\4\0\3\5\f\b\8\9\2\d\1\a\c\2\9\0\c\5\8\1\9\2\f\4\f\8\f ]] 2024-08-25T20:03:28.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:03:28.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 42588 2024-08-25T20:03:29.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=04c0f94ca37da0ff838737dca8cb4ae1 2024-08-25T20:03:29.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-08-25T20:03:29.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 04c0f94ca37da0ff838737dca8cb4ae1 != \0\4\c\0\f\9\4\c\a\3\7\d\a\0\f\f\8\3\8\7\3\7\d\c\a\8\c\b\4\a\e\1 ]] 2024-08-25T20:03:29.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=52540cb803b22fe1b11f84adeb12870d 2024-08-25T20:03:29.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-08-25T20:03:29.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=0b46a7817364e0d1d142a7c1974b3310 2024-08-25T20:03:29.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-08-25T20:03:29.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 52540cb803b22fe1b11f84adeb12870d != \5\2\5\4\0\c\b\8\0\3\b\2\2\f\e\1\b\1\1\f\8\4\a\d\e\b\1\2\8\7\0\d ]] 2024-08-25T20:03:30.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 0b46a7817364e0d1d142a7c1974b3310 != \0\b\4\6\a\7\8\1\7\3\6\4\e\0\d\1\d\1\4\2\a\7\c\1\9\7\4\b\3\3\1\0 ]] 2024-08-25T20:03:30.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=098d2d1f751acef337ce5410722cc188 2024-08-25T20:03:30.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-08-25T20:03:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 098d2d1f751acef337ce5410722cc188 != \0\9\8\d\2\d\1\f\7\5\1\a\c\e\f\3\3\7\c\e\5\4\1\0\7\2\2\c\c\1\8\8 ]] 2024-08-25T20:03:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:03:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 42590 2024-08-25T20:03:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:03:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 42592 2024-08-25T20:03:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:03:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 42594 2024-08-25T20:03:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:03:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 42596 2024-08-25T20:03:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:03:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 42598 2024-08-25T20:03:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:03:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 42600 2024-08-25T20:03:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:03:30.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 42602 2024-08-25T20:03:30.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=9b09423049a10597d61bd679ae821407 2024-08-25T20:03:30.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-08-25T20:03:30.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 9b09423049a10597d61bd679ae821407 != \9\b\0\9\4\2\3\0\4\9\a\1\0\5\9\7\d\6\1\b\d\6\7\9\a\e\8\2\1\4\0\7 ]] 2024-08-25T20:03:30.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:03:30.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:30.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:30.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:03:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-08-25T20:03:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:03:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:32.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:32.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:32.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:32.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:03:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:03:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4c2ba3a4-f815-4f35-86fb-0f609016cfb2 2024-08-25T20:03:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:03:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3282534.4,"bytes_per_snapshot":65650688.0,"last_snapshot_bytes":65650688,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616151,"remote_snapshot_timestamp":1724616151,"replay_state":"idle"} 2024-08-25T20:03:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:03:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:03:28' 2024-08-25T20:03:32.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:03:32.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:03:32.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:03:32.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3282534.4,"bytes_per_snapshot":65650688.0,"last_snapshot_bytes":65650688,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616151,"remote_snapshot_timestamp":1724616151,"replay_state":"idle"} 2024-08-25T20:03:32.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:03:32.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:32.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:03:32.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:32.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:03:32.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:03:32.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:03:32.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:03:32.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:32.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-08-25 20:03:28' 2024-08-25T20:03:32.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:32.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:32.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:03:32.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:32.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:03:28' 2024-08-25T20:03:32.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:32.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:32.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:03:32.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14c2ba3a4-f815-4f35-86fb-0f609016cfb2up+replayingreplaying, {"bytes_per_second":3282534.4,"bytes_per_snapshot":65650688.0,"last_snapshot_bytes":65650688,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616151,"remote_snapshot_timestamp":1724616151,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+replayingreplaying, {"bytes_per_second":3176448.0,"bytes_per_snapshot":63528960.0,"last_snapshot_bytes":63528960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+replayingreplaying, {"bytes_per_second":3420364.8,"bytes_per_snapshot":68407296.0,"last_snapshot_bytes":68407296,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+replayingreplaying, {"bytes_per_second":3330048.0,"bytes_per_snapshot":66600960.0,"last_snapshot_bytes":66600960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616156,"remote_snapshot_timestamp":1724616156,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+replayingreplaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+replayingreplaying, {"bytes_per_second":3428761.6,"bytes_per_snapshot":68575232.0,"last_snapshot_bytes":68575232,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616151,"remote_snapshot_timestamp":1724616151,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+replayingreplaying, {"bytes_per_second":3334963.2,"bytes_per_snapshot":66699264.0,"last_snapshot_bytes":66699264,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+replayingreplaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+replayingreplaying, {"bytes_per_second":3232153.6,"bytes_per_snapshot":64643072.0,"last_snapshot_bytes":64643072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28' 2024-08-25T20:03:32.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:03:32.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:03:32.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:32.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:03:32.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:03:32.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:32.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:03:32.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:32.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:32.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:32.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary1 2024-08-25T20:03:32.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:03:32.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:32.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:03:32.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-08-25T20:03:32.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-08-25T20:03:32.442 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:32.441+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade6a1a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:03:32.469 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:32.441+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555adf432000 [3/4c2ba3a4-f815-4f35-86fb-0f609016cfb2] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:03:34.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:03:34.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-08-25T20:03:34.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:34.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:34.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:03:34.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:34.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:03:34.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary1 2024-08-25T20:03:34.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:34.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary1 2024-08-25T20:03:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:03:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:03:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-08-25T20:03:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:03:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:03:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-08-25T20:03:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:03:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:03:35.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-08-25T20:03:35.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:35.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:35.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:03:35.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:35.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:35.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:35.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:35.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:35.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:03:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:03:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4b479bdf-9f6e-4198-8e79-ff93a2d21c04 2024-08-25T20:03:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:03:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3420364.8,"bytes_per_snapshot":68407296.0,"last_snapshot_bytes":68407296,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"} 2024-08-25T20:03:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:03:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:03:28' 2024-08-25T20:03:35.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:03:35.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:03:35.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:03:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3420364.8,"bytes_per_snapshot":68407296.0,"last_snapshot_bytes":68407296,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"} 2024-08-25T20:03:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:03:35.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:35.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:03:35.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:35.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:03:35.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:03:35.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:03:35.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:03:35.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:35.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-08-25 20:03:28' 2024-08-25T20:03:35.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:35.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:35.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:03:35.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:35.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:03:28' 2024-08-25T20:03:35.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:35.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:35.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:03:35.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING941904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+replayingreplaying, {"bytes_per_second":3176448.0,"bytes_per_snapshot":63528960.0,"last_snapshot_bytes":63528960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary24b479bdf-9f6e-4198-8e79-ff93a2d21c04up+replayingreplaying, {"bytes_per_second":3420364.8,"bytes_per_snapshot":68407296.0,"last_snapshot_bytes":68407296,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+replayingreplaying, {"bytes_per_second":3330048.0,"bytes_per_snapshot":66600960.0,"last_snapshot_bytes":66600960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616156,"remote_snapshot_timestamp":1724616156,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+replayingreplaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+replayingreplaying, {"bytes_per_second":3428761.6,"bytes_per_snapshot":68575232.0,"last_snapshot_bytes":68575232,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616151,"remote_snapshot_timestamp":1724616151,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+replayingreplaying, {"bytes_per_second":3334963.2,"bytes_per_snapshot":66699264.0,"last_snapshot_bytes":66699264,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+replayingreplaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+replayingreplaying, {"bytes_per_second":3232153.6,"bytes_per_snapshot":64643072.0,"last_snapshot_bytes":64643072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28' 2024-08-25T20:03:35.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:03:35.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:03:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:35.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:03:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:03:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:35.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:03:35.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:35.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:35.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:35.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary2 2024-08-25T20:03:35.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:03:35.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:35.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:03:35.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-08-25T20:03:35.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-08-25T20:03:35.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:35.488+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adf01dba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:03:35.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:35.488+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972000 [3/4b479bdf-9f6e-4198-8e79-ff93a2d21c04] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:03:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:03:37.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-08-25T20:03:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:03:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:03:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary2 2024-08-25T20:03:37.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:37.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary2 2024-08-25T20:03:37.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:03:37.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:03:37.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-08-25T20:03:37.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:37.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:37.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:03:37.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:03:37.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-08-25T20:03:37.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:37.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:37.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:03:37.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:37.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:37.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:37.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:37.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:03:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-08-25T20:03:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:03:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:03:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:03:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6a6a0d98-0015-4435-a50c-afd86813e19e 2024-08-25T20:03:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:03:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3330048.0,"bytes_per_snapshot":66600960.0,"last_snapshot_bytes":66600960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616156,"remote_snapshot_timestamp":1724616156,"replay_state":"idle"} 2024-08-25T20:03:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:03:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:03:28' 2024-08-25T20:03:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:03:38.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:03:38.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:03:38.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3330048.0,"bytes_per_snapshot":66600960.0,"last_snapshot_bytes":66600960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616156,"remote_snapshot_timestamp":1724616156,"replay_state":"idle"} 2024-08-25T20:03:38.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:03:38.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:38.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:03:38.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:38.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:03:38.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:03:38.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:03:38.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:03:38.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:38.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-08-25 20:03:28' 2024-08-25T20:03:38.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:38.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:38.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:03:38.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:38.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:03:28' 2024-08-25T20:03:38.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:38.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:38.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:03:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING841904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+replayingreplaying, {"bytes_per_second":3176448.0,"bytes_per_snapshot":63528960.0,"last_snapshot_bytes":63528960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary36a6a0d98-0015-4435-a50c-afd86813e19eup+replayingreplaying, {"bytes_per_second":3330048.0,"bytes_per_snapshot":66600960.0,"last_snapshot_bytes":66600960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616156,"remote_snapshot_timestamp":1724616156,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+replayingreplaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+replayingreplaying, {"bytes_per_second":3428761.6,"bytes_per_snapshot":68575232.0,"last_snapshot_bytes":68575232,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616151,"remote_snapshot_timestamp":1724616151,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+replayingreplaying, {"bytes_per_second":3334963.2,"bytes_per_snapshot":66699264.0,"last_snapshot_bytes":66699264,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+replayingreplaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+replayingreplaying, {"bytes_per_second":3232153.6,"bytes_per_snapshot":64643072.0,"last_snapshot_bytes":64643072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28' 2024-08-25T20:03:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:03:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:03:38.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:38.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:03:38.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:03:38.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:38.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:03:38.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:38.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:38.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:38.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary3 2024-08-25T20:03:38.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:03:38.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:38.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:03:38.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-08-25T20:03:38.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-08-25T20:03:38.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:38.483+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adfb67520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:03:38.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:38.483+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973b80 [3/6a6a0d98-0015-4435-a50c-afd86813e19e] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:03:39.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:03:39.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-08-25T20:03:39.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:39.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:39.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:03:39.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:39.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:03:39.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary3 2024-08-25T20:03:39.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:39.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary3 2024-08-25T20:03:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:03:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:03:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-08-25T20:03:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:03:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:03:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-08-25T20:03:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:03:39.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:39.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:39.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:39.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:39.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:03:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-08-25T20:03:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:03:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:03:40.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:03:40.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5f5a643b-a56b-4251-aa94-d881dc6cc262 2024-08-25T20:03:40.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:03:40.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"} 2024-08-25T20:03:40.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:03:40.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:03:28' 2024-08-25T20:03:40.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:03:40.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:03:40.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:03:40.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"} 2024-08-25T20:03:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:03:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:03:40.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:40.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:03:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:03:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:03:40.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:03:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-08-25 20:03:28' 2024-08-25T20:03:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:03:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:03:28' 2024-08-25T20:03:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:03:40.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING741904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+replayingreplaying, {"bytes_per_second":3176448.0,"bytes_per_snapshot":63528960.0,"last_snapshot_bytes":63528960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary45f5a643b-a56b-4251-aa94-d881dc6cc262up+replayingreplaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+replayingreplaying, {"bytes_per_second":3428761.6,"bytes_per_snapshot":68575232.0,"last_snapshot_bytes":68575232,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616151,"remote_snapshot_timestamp":1724616151,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+replayingreplaying, {"bytes_per_second":3334963.2,"bytes_per_snapshot":66699264.0,"last_snapshot_bytes":66699264,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+replayingreplaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+replayingreplaying, {"bytes_per_second":3232153.6,"bytes_per_snapshot":64643072.0,"last_snapshot_bytes":64643072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28' 2024-08-25T20:03:40.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:03:40.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:03:40.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:40.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:03:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:03:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:40.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:03:40.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:40.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:40.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary4 2024-08-25T20:03:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:03:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:03:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-08-25T20:03:41.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-08-25T20:03:41.185 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:41.184+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adfb66340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:03:41.185 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:41.184+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972780 [3/5f5a643b-a56b-4251-aa94-d881dc6cc262] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:03:42.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:03:42.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-08-25T20:03:42.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:42.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:42.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:03:42.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:42.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:03:42.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary4 2024-08-25T20:03:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary4 2024-08-25T20:03:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:03:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:03:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-08-25T20:03:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:03:42.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:03:42.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-08-25T20:03:42.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:42.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:42.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:03:42.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:42.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:42.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:42.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:42.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:03:43.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-08-25T20:03:43.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:43.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:43.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:03:43.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:43.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:43.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:43.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:43.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:43.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:03:43.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:03:43.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d07b684d-ae92-495b-af63-7a0de0e70d08 2024-08-25T20:03:43.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:03:43.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3428761.6,"bytes_per_snapshot":68575232.0,"last_snapshot_bytes":68575232,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616151,"remote_snapshot_timestamp":1724616151,"replay_state":"idle"} 2024-08-25T20:03:43.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:03:43.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:03:28' 2024-08-25T20:03:43.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:03:43.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:03:43.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:03:43.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3428761.6,"bytes_per_snapshot":68575232.0,"last_snapshot_bytes":68575232,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616151,"remote_snapshot_timestamp":1724616151,"replay_state":"idle"} 2024-08-25T20:03:43.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:03:43.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:43.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:03:43.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:43.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:03:43.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:03:43.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:03:43.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:03:43.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:43.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-08-25 20:03:28' 2024-08-25T20:03:43.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:43.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:43.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:03:43.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:43.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:03:28' 2024-08-25T20:03:43.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:43.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:43.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:03:43.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING641904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+replayingreplaying, {"bytes_per_second":3176448.0,"bytes_per_snapshot":63528960.0,"last_snapshot_bytes":63528960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary5d07b684d-ae92-495b-af63-7a0de0e70d08up+replayingreplaying, {"bytes_per_second":3428761.6,"bytes_per_snapshot":68575232.0,"last_snapshot_bytes":68575232,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616151,"remote_snapshot_timestamp":1724616151,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+replayingreplaying, {"bytes_per_second":3334963.2,"bytes_per_snapshot":66699264.0,"last_snapshot_bytes":66699264,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+replayingreplaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+replayingreplaying, {"bytes_per_second":3232153.6,"bytes_per_snapshot":64643072.0,"last_snapshot_bytes":64643072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28' 2024-08-25T20:03:43.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:03:43.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:03:44.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:44.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:03:44.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:03:44.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:44.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:03:44.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:44.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:44.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:44.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary5 2024-08-25T20:03:44.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:03:44.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:44.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:03:44.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-08-25T20:03:44.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-08-25T20:03:44.193 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:44.192+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae387bba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:03:44.193 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:44.192+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972c80 [3/d07b684d-ae92-495b-af63-7a0de0e70d08] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:03:45.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:03:45.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-08-25T20:03:45.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:45.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:45.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:03:45.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:45.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:03:45.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary5 2024-08-25T20:03:45.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:45.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary5 2024-08-25T20:03:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:03:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:03:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-08-25T20:03:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:03:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:03:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-08-25T20:03:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:03:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:45.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:45.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:45.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:45.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:03:46.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-08-25T20:03:46.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:46.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:46.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:03:46.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:46.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:46.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:46.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:46.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:46.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:03:46.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:03:46.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ba93f24c-de3f-4b9f-9f95-a8a9c5aa159d 2024-08-25T20:03:46.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:03:46.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3334963.2,"bytes_per_snapshot":66699264.0,"last_snapshot_bytes":66699264,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"} 2024-08-25T20:03:46.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:03:46.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:03:28' 2024-08-25T20:03:46.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:03:46.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:03:46.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:03:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3334963.2,"bytes_per_snapshot":66699264.0,"last_snapshot_bytes":66699264,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"} 2024-08-25T20:03:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:03:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:03:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:03:46.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:03:46.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:03:46.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:03:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-08-25 20:03:28' 2024-08-25T20:03:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:46.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:03:46.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:46.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:03:28' 2024-08-25T20:03:46.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:46.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:46.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:03:47.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING541904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+replayingreplaying, {"bytes_per_second":3176448.0,"bytes_per_snapshot":63528960.0,"last_snapshot_bytes":63528960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary6ba93f24c-de3f-4b9f-9f95-a8a9c5aa159dup+replayingreplaying, {"bytes_per_second":3334963.2,"bytes_per_snapshot":66699264.0,"last_snapshot_bytes":66699264,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+replayingreplaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+replayingreplaying, {"bytes_per_second":3232153.6,"bytes_per_snapshot":64643072.0,"last_snapshot_bytes":64643072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28' 2024-08-25T20:03:47.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:03:47.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:03:47.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:47.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:03:47.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:03:47.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:47.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:03:47.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:47.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:47.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:47.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary6 2024-08-25T20:03:47.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:03:47.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:47.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:03:47.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-08-25T20:03:47.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-08-25T20:03:47.252 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:47.251+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae387bd40 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:03:47.252 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:47.251+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972500 [3/ba93f24c-de3f-4b9f-9f95-a8a9c5aa159d] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:03:48.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:03:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-08-25T20:03:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:03:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:03:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary6 2024-08-25T20:03:48.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:48.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary6 2024-08-25T20:03:48.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:03:48.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:03:48.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-08-25T20:03:48.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:48.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:48.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:03:48.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:03:48.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-08-25T20:03:48.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:48.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:48.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:03:48.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:48.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:48.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:48.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:48.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:03:49.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-08-25T20:03:49.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:49.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:49.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:03:49.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:49.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:49.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:49.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:49.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:49.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:03:49.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:03:49.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 12419548-8f80-4684-838b-1b7f43caadcb 2024-08-25T20:03:49.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:03:49.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"} 2024-08-25T20:03:49.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:03:49.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:03:28' 2024-08-25T20:03:49.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:03:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:03:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:03:49.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"} 2024-08-25T20:03:49.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:03:49.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:49.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:03:49.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:49.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:03:49.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:03:49.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:03:49.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:03:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-08-25 20:03:28' 2024-08-25T20:03:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:49.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:03:49.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:49.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:03:28' 2024-08-25T20:03:49.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:49.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:49.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:03:49.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING441904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+replayingreplaying, {"bytes_per_second":3176448.0,"bytes_per_snapshot":63528960.0,"last_snapshot_bytes":63528960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary712419548-8f80-4684-838b-1b7f43caadcbup+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+replayingreplaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+replayingreplaying, {"bytes_per_second":3232153.6,"bytes_per_snapshot":64643072.0,"last_snapshot_bytes":64643072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28' 2024-08-25T20:03:49.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:03:49.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:03:49.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:49.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:03:49.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:03:49.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:50.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:03:50.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:50.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:50.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:50.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary7 2024-08-25T20:03:50.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:03:50.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:50.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:03:50.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-08-25T20:03:50.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-08-25T20:03:50.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:50.189+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adfb664e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:03:50.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:50.189+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972f00 [3/12419548-8f80-4684-838b-1b7f43caadcb] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:03:51.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:03:51.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-08-25T20:03:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:03:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:03:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary7 2024-08-25T20:03:51.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:51.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary7 2024-08-25T20:03:51.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:03:51.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:03:51.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-08-25T20:03:51.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:51.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:51.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:03:51.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:03:51.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-08-25T20:03:51.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:51.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:51.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:03:51.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:51.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:51.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:51.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:51.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:03:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-08-25T20:03:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:03:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:52.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:52.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:52.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:52.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:52.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:03:52.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:03:52.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6836838d-c369-4977-8900-df8dc7a72cf6 2024-08-25T20:03:52.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:03:52.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"} 2024-08-25T20:03:52.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:03:52.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:03:28' 2024-08-25T20:03:52.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:03:52.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:03:52.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:03:52.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"} 2024-08-25T20:03:52.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:03:52.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:52.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:03:52.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:52.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:03:52.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:03:52.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:03:52.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:03:52.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:52.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-08-25 20:03:28' 2024-08-25T20:03:52.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:52.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:52.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:03:52.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:52.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:03:28' 2024-08-25T20:03:52.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:52.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:52.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:03:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING341904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+replayingreplaying, {"bytes_per_second":3176448.0,"bytes_per_snapshot":63528960.0,"last_snapshot_bytes":63528960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary86836838d-c369-4977-8900-df8dc7a72cf6up+replayingreplaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616157,"remote_snapshot_timestamp":1724616157,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+replayingreplaying, {"bytes_per_second":3232153.6,"bytes_per_snapshot":64643072.0,"last_snapshot_bytes":64643072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28' 2024-08-25T20:03:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:03:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:03:52.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:53.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:03:53.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:03:53.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:53.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:03:53.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:53.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:53.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:53.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary8 2024-08-25T20:03:53.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:03:53.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:53.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:03:53.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-08-25T20:03:53.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-08-25T20:03:53.199 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:53.198+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adfb67380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:03:53.199 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:53.198+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555adf432500 [3/6836838d-c369-4977-8900-df8dc7a72cf6] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:03:54.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:03:54.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-08-25T20:03:54.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:54.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:54.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:03:54.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:54.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:03:54.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary8 2024-08-25T20:03:54.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:54.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary8 2024-08-25T20:03:54.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:03:54.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:03:54.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-08-25T20:03:54.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:54.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:54.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:03:54.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:03:54.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-08-25T20:03:54.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:54.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:54.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:03:54.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:54.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:54.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:54.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:54.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:03:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-08-25T20:03:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:03:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:55.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:55.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:55.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:55.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:55.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:03:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:03:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 44ece9fc-ce13-4f35-b342-6d5322202441 2024-08-25T20:03:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:03:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3232153.6,"bytes_per_snapshot":64643072.0,"last_snapshot_bytes":64643072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"} 2024-08-25T20:03:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:03:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:03:28' 2024-08-25T20:03:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:03:55.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:03:55.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:03:55.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3232153.6,"bytes_per_snapshot":64643072.0,"last_snapshot_bytes":64643072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"} 2024-08-25T20:03:55.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:03:55.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:55.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:03:55.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:55.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:03:55.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:03:55.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:03:55.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:03:55.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:55.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-08-25 20:03:28' 2024-08-25T20:03:55.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:55.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:55.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:03:55.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:55.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:03:28' 2024-08-25T20:03:55.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:55.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:55.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:03:56.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING241904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+replayingreplaying, {"bytes_per_second":3176448.0,"bytes_per_snapshot":63528960.0,"last_snapshot_bytes":63528960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28image-primary944ece9fc-ce13-4f35-b342-6d5322202441up+replayingreplaying, {"bytes_per_second":3232153.6,"bytes_per_snapshot":64643072.0,"last_snapshot_bytes":64643072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616153,"remote_snapshot_timestamp":1724616153,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:28' 2024-08-25T20:03:56.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:03:56.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:03:56.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:28' 2024-08-25T20:03:56.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:03:56.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:03:56.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:56.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:03:56.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:56.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:56.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:56.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary9 2024-08-25T20:03:56.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:03:56.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:56.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:03:56.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-08-25T20:03:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-08-25T20:03:56.304 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:56.303+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8ba4e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:03:56.304 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:56.303+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972280 [3/44ece9fc-ce13-4f35-b342-6d5322202441] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:03:58.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:03:58.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-08-25T20:03:58.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:58.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:58.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:03:58.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:58.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:03:58.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary9 2024-08-25T20:03:58.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:58.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary9 2024-08-25T20:03:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:03:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:03:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-08-25T20:03:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:03:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:03:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-08-25T20:03:58.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:58.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:58.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:03:58.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:58.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:58.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:58.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:03:58.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:03:59.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-08-25T20:03:59.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:59.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:59.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:03:59.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:59.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:03:59.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:03:59.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:59.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:59.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:03:59.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:03:59.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f6047f7-e198-4c21-aeb1-fcc11903a8a7 2024-08-25T20:03:59.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:03:59.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63528960.0,"last_snapshot_bytes":63528960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"} 2024-08-25T20:03:59.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:03:59.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:03:58' 2024-08-25T20:03:59.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:03:59.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:03:59.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:03:59.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63528960.0,"last_snapshot_bytes":63528960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"} 2024-08-25T20:03:59.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:03:59.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:59.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:03:59.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:59.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:03:59.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:03:59.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:03:59.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:03:59.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:58' 2024-08-25T20:03:59.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-08-25 20:03:58' 2024-08-25T20:03:59.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:03:59.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:59.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:03:59.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:03:59.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:03:58' 2024-08-25T20:03:59.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:03:59.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:03:59.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:03:59.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING141904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary100f6047f7-e198-4c21-aeb1-fcc11903a8a7up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63528960.0,"last_snapshot_bytes":63528960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616158,"remote_snapshot_timestamp":1724616158,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:03:58' 2024-08-25T20:03:59.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:03:59.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:03:59.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:03:58' 2024-08-25T20:03:59.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:03:59.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:03:59.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:03:59.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:03:59.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:03:59.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:59.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:03:59.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary10 2024-08-25T20:03:59.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:03:59.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:03:59.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:03:59.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-08-25T20:03:59.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-08-25T20:03:59.458 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:59.457+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade2d7040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:03:59.458 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:03:59.457+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ae4fe2a00 [3/0f6047f7-e198-4c21-aeb1-fcc11903a8a7] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:04:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:04:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-08-25T20:04:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:04:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:04:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary10 2024-08-25T20:04:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:04:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary10 2024-08-25T20:04:00.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..10} 2024-08-25T20:04:00.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS=() 2024-08-25T20:04:00.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS=() 2024-08-25T20:04:00.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS=() 2024-08-25T20:04:00.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS=() 2024-08-25T20:04:00.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RET=0 2024-08-25T20:04:00.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:04:00.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:04:00.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary1 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:00.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:00.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:00.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:04:00.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:04:00.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:04:00.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:00.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-08-25T20:04:00.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:04:00.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:00.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:04:00.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:04:00.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-08-25T20:04:00.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:00.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:00.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:00.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:00.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-08-25T20:04:01.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:04:01.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-08-25T20:04:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary1': 2024-08-25T20:04:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:04:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:04:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:04:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1aa8c2fc73ff 2024-08-25T20:04:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1aa8c2fc73ff 2024-08-25T20:04:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:04:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:04:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:04:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:04:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:04:00 2024 2024-08-25T20:04:01.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:04:00 2024 2024-08-25T20:04:01.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:04:00 2024 2024-08-25T20:04:01.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:04:01.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:04:01.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 40fe2c69-4e1c-45c2-8f78-7644b7280ef4 2024-08-25T20:04:01.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:04:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:04:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:04:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-08-25T20:04:01.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd0 2024-08-25T20:04:01.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:04:01.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-08-25T20:04:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:04:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:04:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:04:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: e7ea0506-631b-4c53-ac75-72aa739b3da3 2024-08-25T20:04:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:04:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:04:01.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:01.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:04:01.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:04:01.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:04:01.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:04:01.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:01.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary1 2024-08-25T20:04:01.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-08-25T20:04:01.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:04:01.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary1 2024-08-25T20:04:01.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:04:01.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-08-25T20:04:01.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:01.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:01.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:01.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:01.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:01.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:01.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:01.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary2 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary1 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:04:01.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:01.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-08-25T20:04:01.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary1 2024-08-25T20:04:01.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-08-25T20:04:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:04:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:04:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:04:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-08-25T20:04:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-08-25T20:04:01.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:04:01.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-08-25T20:04:02.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:04:02.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 664 2024-08-25T20:04:02.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-08-25T20:04:02.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary2': 2024-08-25T20:04:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:04:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:04:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:04:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1ab81f85c01f 2024-08-25T20:04:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1ab81f85c01f 2024-08-25T20:04:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:04:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:04:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:04:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:04:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:04:01 2024 2024-08-25T20:04:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:04:01 2024 2024-08-25T20:04:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:04:01 2024 2024-08-25T20:04:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:04:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:04:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: e04d9be2-9a9b-45f2-aa7b-58edbddae93a 2024-08-25T20:04:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:04:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:04:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:04:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-08-25T20:04:02.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd1 2024-08-25T20:04:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:04:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-08-25T20:04:02.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:04:02.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:04:02.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:04:02.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 74dd10be-f40a-43e3-a48d-0d82237acd7d 2024-08-25T20:04:02.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:04:02.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:04:02.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:02.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:04:02.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:04:02.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:04:02.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:04:02.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:02.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary2 2024-08-25T20:04:02.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-08-25T20:04:02.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:04:02.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary2 2024-08-25T20:04:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:04:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-08-25T20:04:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary2 2024-08-25T20:04:02.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary3 2024-08-25T20:04:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-08-25T20:04:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:02.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:04:02.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary2 2024-08-25T20:04:02.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:02.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-08-25T20:04:02.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-08-25T20:04:02.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:02.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:02.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:02.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:02.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:02.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:04:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:04:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:04:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-08-25T20:04:02.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:04:02.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:02.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:04:02.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:04:02.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-08-25T20:04:02.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:02.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:02.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:02.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:02.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-08-25T20:04:02.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:04:02.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-08-25T20:04:03.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:04:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 666 2024-08-25T20:04:03.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-08-25T20:04:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary3': 2024-08-25T20:04:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:04:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:04:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1ac7587f79e1 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1ac7587f79e1 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:04:02 2024 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:04:02 2024 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:04:02 2024 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 96ee44c8-c5eb-4e4b-9f36-cd4c135c2610 2024-08-25T20:04:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:04:03.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:04:03.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:04:03.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-08-25T20:04:03.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd2 2024-08-25T20:04:03.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:04:03.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-08-25T20:04:03.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:04:03.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:04:03.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:04:03.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 80958cf2-98be-44cd-8e21-a3e8b6094e0e 2024-08-25T20:04:03.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:04:03.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:04:03.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:03.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:04:03.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:04:03.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:04:03.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:04:03.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:03.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary3 2024-08-25T20:04:03.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-08-25T20:04:03.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:04:03.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary3 2024-08-25T20:04:03.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:04:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-08-25T20:04:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:03.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:03.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:04:03.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:04:03.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:04:03.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary4 2024-08-25T20:04:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-08-25T20:04:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary3 2024-08-25T20:04:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:04:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary3 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:03.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-08-25T20:04:03.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:04:03.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:03.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:04:03.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:04:03.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-08-25T20:04:03.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:03.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:03.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:03.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:03.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-08-25T20:04:04.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:04:04.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-08-25T20:04:04.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:04:04.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 668 2024-08-25T20:04:04.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-08-25T20:04:04.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary4': 2024-08-25T20:04:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:04:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:04:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:04:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1ad52230b381 2024-08-25T20:04:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1ad52230b381 2024-08-25T20:04:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:04:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:04:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:04:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:04:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:04:03 2024 2024-08-25T20:04:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:04:03 2024 2024-08-25T20:04:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:04:03 2024 2024-08-25T20:04:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:04:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:04:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 6113c176-30b4-41f0-a3f8-995119a61317 2024-08-25T20:04:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:04:04.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:04:04.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:04:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-08-25T20:04:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd3 2024-08-25T20:04:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:04:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-08-25T20:04:04.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:04:04.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:04:04.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:04:04.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: cc9f5666-503a-4a51-ba64-7549805c9e5d 2024-08-25T20:04:04.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:04:04.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:04:04.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:04.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:04:04.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:04:04.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:04:04.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:04:04.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:04.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary4 2024-08-25T20:04:04.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-08-25T20:04:04.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:04:04.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary4 2024-08-25T20:04:04.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:04:04.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:04:04.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:04:04.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:04:04.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary5 2024-08-25T20:04:04.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-08-25T20:04:04.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary4 2024-08-25T20:04:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:04:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-08-25T20:04:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:04.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:04.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:04.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:04.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:04.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:04.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:04:04.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:04:04.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:04:04.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:04.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-08-25T20:04:04.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary4 2024-08-25T20:04:04.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-08-25T20:04:04.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-08-25T20:04:04.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:04.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:04.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:04.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:04.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:04:04.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:04.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:04.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:04.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:04:04.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:04:04.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:04.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:04:04.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:04:04.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-08-25T20:04:04.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:04.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:04.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:04.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:04.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-08-25T20:04:04.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:04:04.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-08-25T20:04:05.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:04:05.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 670 2024-08-25T20:04:05.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-08-25T20:04:05.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:05.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:05.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:05.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:05.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:05.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:05.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:05.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary5': 2024-08-25T20:04:05.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:04:05.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:04:05.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:04:05.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1ae29eb31528 2024-08-25T20:04:05.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1ae29eb31528 2024-08-25T20:04:05.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:04:05.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:04:05.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:04:05.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:04:05.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:04:04 2024 2024-08-25T20:04:05.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:04:04 2024 2024-08-25T20:04:05.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:04:04 2024 2024-08-25T20:04:05.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:04:05.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:04:05.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: dfe57283-5a78-4b59-8ca5-7585d4556e53 2024-08-25T20:04:05.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:04:05.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:04:05.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:04:05.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-08-25T20:04:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd4 2024-08-25T20:04:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:04:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-08-25T20:04:05.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:04:05.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:04:05.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:04:05.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 5535b294-10cf-4d01-9211-780d5b61c914 2024-08-25T20:04:05.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:04:05.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:04:05.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:05.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:04:05.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:04:05.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:04:05.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:04:05.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:05.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary5 2024-08-25T20:04:05.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-08-25T20:04:05.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:04:05.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary5 2024-08-25T20:04:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:04:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-08-25T20:04:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:04:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary6 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary5 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary5 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:04:05.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:04:05.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:05.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-08-25T20:04:05.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:04:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:04:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:04:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-08-25T20:04:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-08-25T20:04:05.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:04:05.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-08-25T20:04:06.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 673 2024-08-25T20:04:06.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:04:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 672 2024-08-25T20:04:06.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:06.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-08-25T20:04:06.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:06.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:06.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:06.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:06.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:06.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary6': 2024-08-25T20:04:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:04:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1af5ce8dc771 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1af5ce8dc771 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:04:05 2024 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:04:05 2024 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:04:05 2024 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: acfcd367-56d4-41f2-9237-93f74e314971 2024-08-25T20:04:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:04:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:04:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:04:06.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-08-25T20:04:06.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd5 2024-08-25T20:04:06.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:04:06.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-08-25T20:04:06.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:04:06.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:04:06.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:04:06.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 5791272c-980f-46c0-8c92-8db9f52032ef 2024-08-25T20:04:06.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:04:06.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:04:06.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:06.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:04:06.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:04:06.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:04:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:04:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:06.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary6 2024-08-25T20:04:06.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-08-25T20:04:06.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:04:06.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary6 2024-08-25T20:04:06.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:04:06.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-08-25T20:04:06.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:06.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:06.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:06.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:06.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:04:06.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:06.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:06.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:06.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:04:06.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary7 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary6 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:06.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary6 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:06.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-08-25T20:04:06.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:04:06.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:06.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:04:06.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:04:06.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-08-25T20:04:06.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:06.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:06.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:06.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:06.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-08-25T20:04:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:04:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-08-25T20:04:07.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:04:07.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 675 2024-08-25T20:04:07.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 676 2024-08-25T20:04:07.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:07.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:07.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-08-25T20:04:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:07.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary7': 2024-08-25T20:04:07.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:04:07.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:04:07.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:04:07.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1b0777a6f9c6 2024-08-25T20:04:07.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1b0777a6f9c6 2024-08-25T20:04:07.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:04:07.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:04:07.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:04:07.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:04:07.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:04:06 2024 2024-08-25T20:04:07.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:04:06 2024 2024-08-25T20:04:07.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:04:06 2024 2024-08-25T20:04:07.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:04:07.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:04:07.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: b74e74d2-ddac-461e-8089-2e96311f9b8c 2024-08-25T20:04:07.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:04:07.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:04:07.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:04:07.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-08-25T20:04:07.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd6 2024-08-25T20:04:07.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:04:07.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-08-25T20:04:07.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:04:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:04:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:04:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 64aa1097-c1e5-47f3-8eb4-30d6424ed783 2024-08-25T20:04:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:04:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:04:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:04:07.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:04:07.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:04:07.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:04:07.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:07.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary7 2024-08-25T20:04:07.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-08-25T20:04:07.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:04:07.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary7 2024-08-25T20:04:07.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:04:07.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-08-25T20:04:07.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:07.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:07.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:07.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:07.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:04:07.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:07.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:07.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary8 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary7 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary7 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:04:07.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:07.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-08-25T20:04:07.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:04:07.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:07.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:04:07.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:04:07.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-08-25T20:04:07.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:07.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:07.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:07.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:07.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-08-25T20:04:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:04:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-08-25T20:04:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:04:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:04:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:04:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 678 2024-08-25T20:04:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 679 2024-08-25T20:04:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:08.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:08.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-08-25T20:04:08.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary8': 2024-08-25T20:04:08.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1b19af832d49 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1b19af832d49 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:04:07 2024 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:04:07 2024 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:04:07 2024 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:04:08.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:04:08.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: ec274d6d-37cd-4bf4-995b-bcac0ad1c987 2024-08-25T20:04:08.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:04:08.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:04:08.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:04:08.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-08-25T20:04:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd7 2024-08-25T20:04:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:04:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-08-25T20:04:08.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:04:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:04:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:04:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 753f5552-852d-4a06-aea2-868216786e88 2024-08-25T20:04:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:04:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:04:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:04:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:04:08.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:04:08.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:04:08.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary8 2024-08-25T20:04:08.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-08-25T20:04:08.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:04:08.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary8 2024-08-25T20:04:08.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:04:08.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary9 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary8 2024-08-25T20:04:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:04:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:04:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:04:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-08-25T20:04:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary8 2024-08-25T20:04:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-08-25T20:04:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:04:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:04:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:04:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-08-25T20:04:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-08-25T20:04:08.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:04:08.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-08-25T20:04:09.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:09.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:04:09.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:09.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:09.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:09.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:04:09.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:09.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:09.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:09.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:09.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:09.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:09.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:04:09.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 681 2024-08-25T20:04:09.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 682 2024-08-25T20:04:09.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:09.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-08-25T20:04:09.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:09.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary9': 2024-08-25T20:04:09.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:04:09.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:04:09.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:04:09.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1b2b53aff62d 2024-08-25T20:04:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1b2b53aff62d 2024-08-25T20:04:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:04:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:04:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:04:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:04:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:04:08 2024 2024-08-25T20:04:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:04:08 2024 2024-08-25T20:04:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:04:08 2024 2024-08-25T20:04:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:04:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:04:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 5eb517fd-e079-4cdb-85ca-27fc4caa8722 2024-08-25T20:04:09.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:04:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:04:09.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:04:09.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-08-25T20:04:09.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd8 2024-08-25T20:04:09.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:04:09.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-08-25T20:04:09.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:04:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:04:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:04:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: c7d3a756-f0dd-4a7f-9f72-594d8f35662e 2024-08-25T20:04:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:04:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:04:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:04:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:04:09.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:04:09.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:04:09.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:09.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary9 2024-08-25T20:04:09.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-08-25T20:04:09.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:04:09.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary9 2024-08-25T20:04:09.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:04:09.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-08-25T20:04:09.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:09.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:09.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:09.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:09.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary10 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary9 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:09.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:09.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:09.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:09.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:09.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:04:09.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:09.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-08-25T20:04:09.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:09.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:09.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary9 2024-08-25T20:04:09.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:09.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:09.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:09.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:09.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:04:09.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:04:09.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-08-25T20:04:09.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:04:09.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:04:09.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-08-25T20:04:09.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:04:09.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:09.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:04:09.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:04:09.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-08-25T20:04:09.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:09.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:09.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:09.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:04:09.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-08-25T20:04:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:04:09.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-08-25T20:04:10.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:10.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:04:10.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:10.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:10.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:10.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:04:10.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:10.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:10.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:10.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:10.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:10.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:10.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:04:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 684 2024-08-25T20:04:10.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 685 2024-08-25T20:04:10.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 686 2024-08-25T20:04:10.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:10.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-08-25T20:04:10.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:10.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary10': 2024-08-25T20:04:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:04:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:04:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:04:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1b40e14684bd 2024-08-25T20:04:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1b40e14684bd 2024-08-25T20:04:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:04:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:04:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:04:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:04:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:04:09 2024 2024-08-25T20:04:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:04:09 2024 2024-08-25T20:04:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:04:09 2024 2024-08-25T20:04:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:04:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:04:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 7dccc3b0-bd7c-48cd-afae-c170c8dcb807 2024-08-25T20:04:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:04:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:04:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:04:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-08-25T20:04:10.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd9 2024-08-25T20:04:10.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:04:10.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-08-25T20:04:10.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:04:10.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:04:10.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:04:10.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: b1f8cb37-add2-4c2b-bd00-002d0cf8355b 2024-08-25T20:04:10.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:04:10.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:04:10.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:10.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:04:10.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:04:10.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:04:10.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:04:10.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:04:10.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary10 2024-08-25T20:04:10.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-08-25T20:04:10.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:04:10.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary10 2024-08-25T20:04:10.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47123 2024-08-25T20:04:10.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary10 2024-08-25T20:04:10.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary10 2024-08-25T20:04:10.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-08-25T20:04:10.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:04:10.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-08-25T20:04:11.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 688 2024-08-25T20:04:11.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 690 2024-08-25T20:04:11.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:11.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:11.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:11.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:11.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:11.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:11.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:11.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:11.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:04:11.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:11.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:11.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:11.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:04:11.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:11.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 689 2024-08-25T20:04:11.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:12.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:12.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:04:12.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:12.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:12.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:12.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:04:12.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:12.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:04:12.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:12.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:12.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:12.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:04:12.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 692 2024-08-25T20:04:12.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 691 2024-08-25T20:04:12.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:12.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:13.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 694 2024-08-25T20:04:13.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:13.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:04:13.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:13.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:13.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:13.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:04:13.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:13.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:13.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:13.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:13.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:13.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:13.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:13.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:04:13.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:13.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:13.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:13.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:04:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 693 2024-08-25T20:04:13.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:13.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:14.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:14.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:04:14.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:14.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:14.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:14.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:04:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:04:14.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:14.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:14.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:14.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:04:14.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:14.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:14.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:14.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:14.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:14.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:14.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 697 2024-08-25T20:04:14.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 696 2024-08-25T20:04:14.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 695 2024-08-25T20:04:14.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:14.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:14.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:15.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 698 2024-08-25T20:04:15.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 699 2024-08-25T20:04:15.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 700 2024-08-25T20:04:15.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:15.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:15.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:15.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:15.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:15.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:04:15.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:15.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:15.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:15.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:15.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:15.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:04:15.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:15.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:15.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:16.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 702 2024-08-25T20:04:16.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 701 2024-08-25T20:04:16.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:16.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:04:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:16.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:04:16.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:16.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:04:16.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:16.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:16.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:16.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:04:17.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 703 2024-08-25T20:04:17.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 704 2024-08-25T20:04:17.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:17.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:17.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:17.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:17.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:17.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:17.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:17.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:17.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:17.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:04:17.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:17.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:17.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:17.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:04:17.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:17.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:04:17.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:17.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:17.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:17.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:04:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:04:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:18.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:04:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:18.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:04:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:18.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:04:18.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 705 2024-08-25T20:04:18.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 706 2024-08-25T20:04:18.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 707 2024-08-25T20:04:18.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:18.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:18.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:19.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:19.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:04:19.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:19.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:19.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:19.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:04:19.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:19.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:19.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:19.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:19.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:19.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:19.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 708 2024-08-25T20:04:19.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 709 2024-08-25T20:04:19.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 710 2024-08-25T20:04:19.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:19.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:19.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:20.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:20.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:04:20.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:20.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:20.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:20.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:04:20.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:20.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:04:20.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:20.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:20.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:20.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:04:20.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 711 2024-08-25T20:04:20.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 712 2024-08-25T20:04:20.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:20.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:21.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 713 2024-08-25T20:04:21.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 714 2024-08-25T20:04:21.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:21.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:21.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:21.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:04:21.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:21.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:21.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:21.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:04:21.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:21.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:04:21.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:21.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:21.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:21.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:04:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:21.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:21.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:21.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:21.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:22.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 715 2024-08-25T20:04:22.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 716 2024-08-25T20:04:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:22.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:22.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:22.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:04:22.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:22.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:22.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:22.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:04:22.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:22.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:04:22.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:22.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:22.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:22.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:04:22.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:22.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:22.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:22.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:22.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:22.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:23.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 718 2024-08-25T20:04:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 719 2024-08-25T20:04:23.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:04:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:04:23.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:24.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 717 2024-08-25T20:04:24.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:04:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:04:24.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:24.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:04:24.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:24.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:24.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:24.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:04:24.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 721 2024-08-25T20:04:24.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:25.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 720 2024-08-25T20:04:25.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:25.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:04:25.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:25.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:25.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:25.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:04:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:04:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:04:25.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:25.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 723 2024-08-25T20:04:25.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 722 2024-08-25T20:04:26.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:26.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 725 2024-08-25T20:04:26.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:26.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:26.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:04:26.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:26.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:26.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:26.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:04:26.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:26.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:04:26.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:26.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:26.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:26.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:04:27.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 724 2024-08-25T20:04:27.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:27.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:27.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:27.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:27.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:27.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:27.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:04:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:04:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 727 2024-08-25T20:04:27.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:28.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 726 2024-08-25T20:04:28.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:28.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:28.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:28.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:28.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:28.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:28.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:28.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 729 2024-08-25T20:04:28.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:28.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:04:28.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:28.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:28.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:28.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:04:28.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:29.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 728 2024-08-25T20:04:29.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:29.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:29.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:29.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:29.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:29.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:29.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:29.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 731 2024-08-25T20:04:29.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:29.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:04:29.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:29.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:29.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:29.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:04:29.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:30.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 730 2024-08-25T20:04:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:04:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:30.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:30.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:04:30.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:30.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 733 2024-08-25T20:04:30.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:30.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:30.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:04:30.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:30.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:30.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:30.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:04:31.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 732 2024-08-25T20:04:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:31.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:31.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:04:31.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:31.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:31.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:31.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:04:31.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 735 2024-08-25T20:04:31.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:32.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 734 2024-08-25T20:04:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:04:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:32.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:04:32.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:32.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 737 2024-08-25T20:04:32.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:32.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:32.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:32.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:32.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:32.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:33.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 736 2024-08-25T20:04:33.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:04:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:04:33.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 739 2024-08-25T20:04:33.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:33.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:33.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:33.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:33.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:33.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:33.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:34.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:34.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:04:34.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:34.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:34.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:34.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:04:34.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 738 2024-08-25T20:04:34.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:34.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:34.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:04:34.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:34.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:34.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:34.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:04:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 742 2024-08-25T20:04:34.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:35.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 740 2024-08-25T20:04:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:04:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:04:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:35.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:35.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:04:35.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:35.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:35.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:35.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:04:35.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 743 2024-08-25T20:04:35.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:36.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 741 2024-08-25T20:04:36.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:36.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:04:36.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:36.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:36.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:36.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:04:36.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:36.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:36.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:04:36.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:36.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:36.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:36.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:04:36.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 745 2024-08-25T20:04:36.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:37.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 744 2024-08-25T20:04:37.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:37.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:37.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:37.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:37.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:37.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:37.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:37.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:04:37.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:37.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:37.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:37.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:04:37.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 747 2024-08-25T20:04:37.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 746 2024-08-25T20:04:38.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:38.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:38.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:38.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:38.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:38.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:38.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:38.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 749 2024-08-25T20:04:38.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:38.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:38.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:04:38.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:38.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:38.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:38.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:04:39.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 748 2024-08-25T20:04:39.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:39.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:39.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:39.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:39.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:39.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:39.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:39.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:39.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:04:39.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:39.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:39.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:39.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:04:39.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 751 2024-08-25T20:04:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:40.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 750 2024-08-25T20:04:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:04:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:04:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 753 2024-08-25T20:04:40.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:40.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:40.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:04:40.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:40.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:40.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:40.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:04:41.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 752 2024-08-25T20:04:41.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:41.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:41.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:04:41.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:41.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:41.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:41.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:04:41.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 755 2024-08-25T20:04:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:41.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:41.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:42.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 754 2024-08-25T20:04:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:42.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:42.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:04:42.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:42.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:42.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:42.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:04:42.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 757 2024-08-25T20:04:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:42.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:43.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 756 2024-08-25T20:04:43.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:43.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:43.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:04:43.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:43.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:43.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:43.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:04:43.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 759 2024-08-25T20:04:43.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:43.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:43.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:43.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:43.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:43.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:43.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:44.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:44.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:04:44.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:44.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:44.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:44.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:04:44.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 758 2024-08-25T20:04:44.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:44.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:04:44.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:44.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:44.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:44.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:04:44.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:44.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 761 2024-08-25T20:04:44.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 760 2024-08-25T20:04:45.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:45.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:04:45.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:45.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:45.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:45.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:04:45.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:45.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 763 2024-08-25T20:04:45.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:45.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:45.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:04:45.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:45.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:45.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:45.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:04:46.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 762 2024-08-25T20:04:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:46.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 765 2024-08-25T20:04:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:46.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:46.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:04:46.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:46.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:46.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:46.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:04:46.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:04:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:04:47.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 764 2024-08-25T20:04:47.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:47.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 767 2024-08-25T20:04:47.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:47.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:47.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:47.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:47.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:47.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:47.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:47.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:47.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:04:47.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:47.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:47.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:47.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:04:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 766 2024-08-25T20:04:48.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:48.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 769 2024-08-25T20:04:48.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:48.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:48.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:48.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:48.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:48.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:48.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:48.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:48.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:04:48.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:48.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:48.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:48.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:04:49.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 768 2024-08-25T20:04:49.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:49.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:49.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:49.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:49.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:49.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:49.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:49.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 771 2024-08-25T20:04:49.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:49.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:04:49.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:49.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:49.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:49.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:04:49.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:50.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 770 2024-08-25T20:04:50.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:50.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:04:50.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:50.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:50.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:50.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:04:50.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:50.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 773 2024-08-25T20:04:50.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:50.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:04:50.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:50.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:50.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:50.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:04:50.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:51.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 772 2024-08-25T20:04:51.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:51.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:51.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:04:51.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:51.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:51.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:51.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:04:51.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 775 2024-08-25T20:04:51.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:51.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:51.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:51.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:51.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:51.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:51.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:52.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 774 2024-08-25T20:04:52.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:52.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:04:52.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:52.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:52.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:52.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:04:52.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:52.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:52.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:52.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:52.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:52.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:52.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:52.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 777 2024-08-25T20:04:52.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:53.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 776 2024-08-25T20:04:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:04:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:04:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:53.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:53.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:53.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:53.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:53.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:53.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:53.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 779 2024-08-25T20:04:53.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:54.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 778 2024-08-25T20:04:54.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:54.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:04:54.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:54.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:54.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:54.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:04:54.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:54.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 781 2024-08-25T20:04:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:54.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:54.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:04:54.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:54.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:54.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:04:54.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:04:55.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 780 2024-08-25T20:04:55.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:55.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:55.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:04:55.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:55.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:55.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:55.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:04:55.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 783 2024-08-25T20:04:55.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:55.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:55.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:04:55.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:55.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:55.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:04:55.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:04:56.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 782 2024-08-25T20:04:56.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:56.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 785 2024-08-25T20:04:56.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:04:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:04:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:04:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:04:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:04:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:04:57.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 784 2024-08-25T20:04:57.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:04:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:04:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:04:57.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:57.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 787 2024-08-25T20:04:57.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:57.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:04:57.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:57.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:57.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:04:57.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:04:57.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:58.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 786 2024-08-25T20:04:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:04:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:04:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:04:58.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:04:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:04:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:04:58.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 789 2024-08-25T20:04:58.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:59.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 788 2024-08-25T20:04:59.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:04:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:04:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:04:59.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:04:59.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:04:59.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:04:59.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:04:59.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:04:59.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:04:59.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:04:59.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 791 2024-08-25T20:04:59.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:00.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 790 2024-08-25T20:05:00.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:00.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:05:00.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:05:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:05:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 793 2024-08-25T20:05:00.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:00.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:05:00.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:00.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:00.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:05:00.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:05:00.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:01.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 792 2024-08-25T20:05:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:01.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 795 2024-08-25T20:05:01.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:01.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:01.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:05:01.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:01.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:01.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:05:01.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:05:01.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:01.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:05:01.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:01.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:01.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:05:01.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:05:02.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 794 2024-08-25T20:05:02.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:02.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:05:02.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:02.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:02.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:05:02.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:05:02.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 797 2024-08-25T20:05:02.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:05:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:05:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:05:03.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 796 2024-08-25T20:05:03.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:03.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:03.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:05:03.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:03.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:03.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:05:03.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:05:03.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 799 2024-08-25T20:05:03.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:03.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:03.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:05:03.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:03.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:03.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:05:03.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:05:04.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:04.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:05:04.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:04.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:04.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:05:04.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:05:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:05:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:05:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:05:04.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 798 2024-08-25T20:05:04.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:04.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 801 2024-08-25T20:05:04.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:05.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 800 2024-08-25T20:05:05.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:05.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:05:05.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:05.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:05.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:05:05.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:05:05.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:05.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 804 2024-08-25T20:05:05.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:05.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:05:05.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:05.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:05.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:05:05.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:05:05.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:06.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 802 2024-08-25T20:05:06.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:06.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 805 2024-08-25T20:05:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:05:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:05:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:05:06.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:06.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:05:06.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:06.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:06.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:05:06.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:05:07.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 803 2024-08-25T20:05:07.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:07.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 807 2024-08-25T20:05:07.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:05:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:05:07.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:05:07.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:07.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:05:07.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:07.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:07.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:05:07.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:05:08.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 806 2024-08-25T20:05:08.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:08.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:05:08.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:08.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:08.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:05:08.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:05:08.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:08.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:08.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:05:08.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:08.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:08.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:05:08.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:05:08.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 809 2024-08-25T20:05:08.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:09.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:09.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:05:09.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:09.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:09.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:05:09.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:05:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:05:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:05:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:05:09.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 808 2024-08-25T20:05:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:09.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 811 2024-08-25T20:05:09.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:10.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 810 2024-08-25T20:05:10.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:05:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:05:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:05:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 814 2024-08-25T20:05:10.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:10.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:05:10.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:10.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:10.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:05:10.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:05:10.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:11.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 812 2024-08-25T20:05:11.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:11.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 815 2024-08-25T20:05:11.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:05:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:05:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:05:11.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:11.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:05:11.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:11.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:11.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:05:11.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:05:12.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 813 2024-08-25T20:05:12.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:12.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 817 2024-08-25T20:05:12.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:12.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:12.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:05:12.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:12.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:12.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:05:12.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:05:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:05:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:05:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:05:13.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 816 2024-08-25T20:05:13.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:13.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:05:13.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:13.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:13.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:05:13.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:05:13.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 819 2024-08-25T20:05:13.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:13.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:13.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:05:13.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:13.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:13.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:05:13.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:05:14.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:14.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:05:14.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:14.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:14.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:05:14.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:05:14.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:05:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:05:14.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:05:14.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 818 2024-08-25T20:05:14.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:14.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 821 2024-08-25T20:05:14.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:15.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:15.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:05:15.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:15.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:15.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:05:15.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:05:15.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:15.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:05:15.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:15.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:15.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:05:15.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:05:15.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 820 2024-08-25T20:05:15.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:15.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 823 2024-08-25T20:05:15.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 822 2024-08-25T20:05:16.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:16.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 825 2024-08-25T20:05:16.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:16.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:16.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:05:16.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:16.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:16.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:05:16.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:05:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:05:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:05:16.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:05:17.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 824 2024-08-25T20:05:17.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:17.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 827 2024-08-25T20:05:17.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:17.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:17.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:05:17.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:17.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:17.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:05:17.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:05:17.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:17.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:05:17.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:17.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:17.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:05:17.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:05:18.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 826 2024-08-25T20:05:18.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:18.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 829 2024-08-25T20:05:18.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:18.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:18.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:05:18.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:18.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:18.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:05:18.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:05:18.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:18.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:05:18.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:18.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:18.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:05:18.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:05:19.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 828 2024-08-25T20:05:19.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:19.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:05:19.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:19.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:19.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:05:19.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:05:19.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:19.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 831 2024-08-25T20:05:19.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:19.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:19.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:05:19.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:19.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:19.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:05:19.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:05:20.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 830 2024-08-25T20:05:20.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:20.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:05:20.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:20.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:20.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:05:20.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:05:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:20.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 833 2024-08-25T20:05:20.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:20.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:20.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:05:20.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:20.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:20.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:05:20.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:05:21.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 832 2024-08-25T20:05:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:21.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:21.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:05:21.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:21.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:21.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:05:21.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:05:21.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 835 2024-08-25T20:05:21.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:21.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:21.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:05:21.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:21.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:21.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:05:21.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:05:22.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 834 2024-08-25T20:05:22.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:22.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:05:22.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:22.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:22.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:05:22.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:05:22.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:22.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 837 2024-08-25T20:05:22.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:22.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:05:22.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:22.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:22.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:05:22.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:05:22.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 836 2024-08-25T20:05:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:05:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:05:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:05:23.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:23.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:23.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:05:23.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:23.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:23.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:05:23.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:05:23.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 839 2024-08-25T20:05:23.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:24.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 838 2024-08-25T20:05:24.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:24.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:05:24.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:24.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:24.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:05:24.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:05:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:24.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:24.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:05:24.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:24.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:24.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:05:24.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:05:24.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 841 2024-08-25T20:05:24.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:25.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 840 2024-08-25T20:05:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:05:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:05:25.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:05:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:05:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:05:25.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:05:25.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 843 2024-08-25T20:05:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:26.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 842 2024-08-25T20:05:26.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:26.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:05:26.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:26.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:26.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:05:26.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:05:26.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:26.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 845 2024-08-25T20:05:26.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:26.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:05:26.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:26.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:26.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:05:26.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:05:26.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:27.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 844 2024-08-25T20:05:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:05:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:05:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:05:27.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:27.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 847 2024-08-25T20:05:27.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:27.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:05:27.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:27.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:27.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:05:27.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:05:27.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:28.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 846 2024-08-25T20:05:28.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:28.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:05:28.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:28.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:28.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:05:28.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:05:28.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:28.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 849 2024-08-25T20:05:28.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:28.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:05:28.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:28.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:28.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:05:28.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:05:28.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 848 2024-08-25T20:05:29.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:29.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:05:29.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:29.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:29.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:05:29.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:05:29.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:29.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:29.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:05:29.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:29.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:29.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:05:29.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:05:29.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 851 2024-08-25T20:05:29.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:30.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 850 2024-08-25T20:05:30.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:30.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:05:30.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:30.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:30.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:05:30.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:05:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:05:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:05:30.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:05:30.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:30.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 853 2024-08-25T20:05:30.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:31.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 852 2024-08-25T20:05:31.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:31.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 855 2024-08-25T20:05:31.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:05:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:31.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:31.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:05:31.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:05:31.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:31.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:05:31.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:31.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:31.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:05:31.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:05:32.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 854 2024-08-25T20:05:32.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:32.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:05:32.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:32.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:32.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:05:32.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:05:32.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:32.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 857 2024-08-25T20:05:32.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:32.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:05:32.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:32.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:32.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:05:32.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:05:32.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:33.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 856 2024-08-25T20:05:33.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:33.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 859 2024-08-25T20:05:33.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:33.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:05:33.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:33.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:33.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:05:33.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:05:33.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:33.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:33.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:05:33.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:33.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:33.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:05:33.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:05:34.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:34.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:05:34.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:34.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:34.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:05:34.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:05:34.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 858 2024-08-25T20:05:34.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:34.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:05:34.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:34.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:34.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:05:34.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:05:34.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:34.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 861 2024-08-25T20:05:34.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:35.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 860 2024-08-25T20:05:35.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:35.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:05:35.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:35.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:35.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:05:35.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:05:35.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:35.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 863 2024-08-25T20:05:35.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:35.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:05:35.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:35.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:35.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:05:35.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:05:35.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:36.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 862 2024-08-25T20:05:36.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:36.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 865 2024-08-25T20:05:36.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:36.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:36.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:05:36.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:36.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:36.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:05:36.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:05:36.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:36.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:05:36.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:36.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:36.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:05:36.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:05:37.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 864 2024-08-25T20:05:37.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:37.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 867 2024-08-25T20:05:37.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:37.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:37.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:05:37.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:37.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:37.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:05:37.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:05:37.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:37.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:05:37.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:37.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:37.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:05:37.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:05:38.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 866 2024-08-25T20:05:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:38.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 869 2024-08-25T20:05:38.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:38.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:38.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:05:38.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:38.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:38.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:05:38.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:05:38.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:38.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:05:38.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:38.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:38.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:05:38.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:05:39.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 868 2024-08-25T20:05:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:05:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:05:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:05:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:39.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 871 2024-08-25T20:05:39.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:39.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:05:39.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:39.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:39.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:05:39.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:05:39.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:40.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 870 2024-08-25T20:05:40.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:40.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:05:40.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:40.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:40.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:05:40.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:05:40.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:40.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 873 2024-08-25T20:05:40.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:40.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:05:40.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:40.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:40.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:05:40.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:05:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:41.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 872 2024-08-25T20:05:41.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:41.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:05:41.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:41.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:41.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:05:41.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:05:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:41.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 875 2024-08-25T20:05:41.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:41.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:41.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:05:41.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:41.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:41.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:05:41.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:05:42.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 874 2024-08-25T20:05:42.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:42.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:05:42.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:42.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:42.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:05:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:05:42.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:42.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 877 2024-08-25T20:05:42.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:42.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:05:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:05:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:05:43.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 876 2024-08-25T20:05:43.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:43.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:05:43.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:43.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:43.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:05:43.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:05:43.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:43.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:43.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:05:43.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:43.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:43.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:05:43.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:05:43.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 879 2024-08-25T20:05:43.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:44.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 878 2024-08-25T20:05:44.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:44.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:05:44.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:44.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:44.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:05:44.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:05:44.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:44.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:44.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:05:44.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:44.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:44.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:05:44.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:05:44.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 881 2024-08-25T20:05:44.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:45.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 880 2024-08-25T20:05:45.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:45.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:05:45.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:45.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:45.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:05:45.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:05:45.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:45.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:45.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:05:45.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:45.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:45.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:05:45.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:05:45.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 883 2024-08-25T20:05:45.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 882 2024-08-25T20:05:46.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:46.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:46.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:05:46.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:46.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:46.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:05:46.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:05:46.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 885 2024-08-25T20:05:46.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:46.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:46.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:05:46.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:46.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:46.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:05:46.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:05:47.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 884 2024-08-25T20:05:47.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:47.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:05:47.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:47.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:47.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:05:47.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:05:47.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:47.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 887 2024-08-25T20:05:47.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:47.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:47.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:05:47.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:47.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:47.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:05:47.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:05:48.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 886 2024-08-25T20:05:48.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:05:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:05:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:05:48.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 889 2024-08-25T20:05:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:48.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:48.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:05:48.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:48.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:48.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:05:48.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:05:49.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 888 2024-08-25T20:05:49.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:49.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:05:49.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:49.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:49.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:05:49.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:05:49.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:49.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:49.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:05:49.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:49.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:49.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:05:49.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:05:49.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 891 2024-08-25T20:05:49.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:50.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 890 2024-08-25T20:05:50.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:50.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:05:50.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:50.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:50.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:05:50.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:05:50.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:50.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:50.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:05:50.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:50.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:50.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:05:50.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:05:50.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 893 2024-08-25T20:05:50.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:51.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 892 2024-08-25T20:05:51.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:51.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:51.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:05:51.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:51.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:51.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:05:51.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:05:51.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 895 2024-08-25T20:05:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:51.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:51.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:05:51.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:51.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:51.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:05:51.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:05:52.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 894 2024-08-25T20:05:52.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:52.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:05:52.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:52.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:52.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:05:52.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:05:52.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:52.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 897 2024-08-25T20:05:52.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:52.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:05:52.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:52.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:52.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:05:52.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:05:52.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:53.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 896 2024-08-25T20:05:53.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:53.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:05:53.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:53.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:53.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:05:53.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:05:53.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:53.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:05:53.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:53.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:53.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:05:53.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:05:53.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 899 2024-08-25T20:05:53.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:05:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:05:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:05:54.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 898 2024-08-25T20:05:54.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:05:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:05:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:05:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:54.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 901 2024-08-25T20:05:54.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:55.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 900 2024-08-25T20:05:55.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:55.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:05:55.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:55.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:55.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:05:55.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:05:55.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:55.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:55.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:05:55.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:55.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:55.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:05:55.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:05:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 903 2024-08-25T20:05:55.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:56.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 902 2024-08-25T20:05:56.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:56.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 905 2024-08-25T20:05:56.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:05:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:56.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:05:56.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:05:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:05:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:05:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:05:57.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 904 2024-08-25T20:05:57.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:57.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 907 2024-08-25T20:05:57.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:57.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:57.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:05:57.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:57.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:57.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:05:57.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:05:57.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:57.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:05:57.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:57.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:57.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:05:57.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:05:58.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 906 2024-08-25T20:05:58.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:58.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 909 2024-08-25T20:05:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:58.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:58.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:05:58.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:58.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:58.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:05:58.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:05:58.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:58.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:05:58.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:58.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:58.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:05:58.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:05:59.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 908 2024-08-25T20:05:59.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:59.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:05:59.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:59.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:59.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:05:59.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:05:59.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:05:59.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 911 2024-08-25T20:05:59.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:05:59.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:05:59.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:05:59.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:05:59.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:05:59.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:05:59.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:00.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 910 2024-08-25T20:06:00.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:00.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:06:00.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:00.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:00.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:06:00.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:06:00.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:06:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:06:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:06:00.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 913 2024-08-25T20:06:00.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:01.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 912 2024-08-25T20:06:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:06:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:06:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:06:01.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 915 2024-08-25T20:06:01.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:06:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:06:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:06:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 914 2024-08-25T20:06:02.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:02.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:06:02.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:02.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:02.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:06:02.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:06:02.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:02.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 917 2024-08-25T20:06:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:06:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:06:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:06:02.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:03.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 916 2024-08-25T20:06:03.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:03.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:06:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:06:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:06:03.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 919 2024-08-25T20:06:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:06:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:06:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:06:03.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:04.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:04.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:06:04.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 918 2024-08-25T20:06:04.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:04.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:04.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:06:04.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:06:04.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:06:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:04.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:06:04.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:06:04.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 921 2024-08-25T20:06:04.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 920 2024-08-25T20:06:05.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:05.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:06:05.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:05.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:05.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:06:05.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:06:05.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:05.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 923 2024-08-25T20:06:05.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:05.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:06:05.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:05.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:05.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:06:05.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:06:05.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:06.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 922 2024-08-25T20:06:06.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:06.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 925 2024-08-25T20:06:06.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:06.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:06:06.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:06.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:06.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:06:06.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:06:06.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:06.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:06.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:06:06.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:06.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:06.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:06:06.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:06:07.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 924 2024-08-25T20:06:07.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:07.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:06:07.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:07.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:07.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:06:07.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:06:07.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:07.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 927 2024-08-25T20:06:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:07.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:07.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:06:07.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:07.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:07.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:06:07.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:06:08.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 926 2024-08-25T20:06:08.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:08.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:06:08.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:08.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:08.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:06:08.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:06:08.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:08.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 930 2024-08-25T20:06:08.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:08.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:06:08.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:08.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:08.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:06:08.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:06:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:09.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:09.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:06:09.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:09.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:09.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:06:09.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:06:09.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 928 2024-08-25T20:06:09.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:06:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:06:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:06:09.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:09.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 931 2024-08-25T20:06:09.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:10.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 929 2024-08-25T20:06:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:06:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:06:10.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:06:10.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:06:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:06:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:06:10.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 933 2024-08-25T20:06:10.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:11.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 932 2024-08-25T20:06:11.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:11.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 935 2024-08-25T20:06:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:06:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:11.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:11.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:06:11.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:06:11.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:11.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:11.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:06:11.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:11.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:11.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:06:11.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:06:12.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 934 2024-08-25T20:06:12.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:12.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:06:12.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:12.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:12.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:06:12.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:06:12.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:12.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:06:12.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:12.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:12.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:06:12.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:06:12.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:12.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 937 2024-08-25T20:06:12.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:13.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 936 2024-08-25T20:06:13.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:13.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:06:13.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:13.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:13.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:06:13.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:06:13.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:13.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 939 2024-08-25T20:06:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:06:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:06:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:06:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:14.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:14.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:06:14.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:14.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:14.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:06:14.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:06:14.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 938 2024-08-25T20:06:14.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:14.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:06:14.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:14.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:14.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:06:14.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:06:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:14.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 941 2024-08-25T20:06:14.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:15.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 940 2024-08-25T20:06:15.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:15.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 943 2024-08-25T20:06:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:06:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:06:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:06:15.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:15.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:15.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:06:15.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:15.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:15.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:06:15.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:06:16.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 942 2024-08-25T20:06:16.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:16.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 945 2024-08-25T20:06:16.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:06:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:06:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:06:16.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:16.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:06:16.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:16.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:16.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:06:16.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:06:17.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 944 2024-08-25T20:06:17.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 947 2024-08-25T20:06:17.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:17.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:17.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:06:17.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:17.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:17.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:06:17.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:06:17.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:17.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:06:17.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:17.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:17.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:06:17.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:06:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 946 2024-08-25T20:06:18.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:18.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 949 2024-08-25T20:06:18.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:18.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:18.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:06:18.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:18.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:18.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:06:18.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:06:18.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:18.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:06:18.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:18.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:18.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:06:18.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:06:19.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 948 2024-08-25T20:06:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:06:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:06:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:06:19.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:19.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 951 2024-08-25T20:06:19.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:19.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:19.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:06:19.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:19.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:19.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:06:19.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:06:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 950 2024-08-25T20:06:20.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:20.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:06:20.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:20.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:20.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:06:20.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:06:20.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:20.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 953 2024-08-25T20:06:20.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:20.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:06:20.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:20.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:20.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:06:20.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:06:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:21.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 952 2024-08-25T20:06:21.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:21.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:06:21.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:21.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:21.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:06:21.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:06:21.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:21.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 955 2024-08-25T20:06:21.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:06:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:06:21.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:06:22.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 954 2024-08-25T20:06:22.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:22.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:06:22.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:22.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:22.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:06:22.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:06:22.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 957 2024-08-25T20:06:22.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:22.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:22.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:06:22.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:22.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:22.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:06:22.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:06:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 956 2024-08-25T20:06:23.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:23.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:06:23.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:23.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:23.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:06:23.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:06:23.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:23.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 959 2024-08-25T20:06:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:06:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:06:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:06:23.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:24.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:24.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:06:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 958 2024-08-25T20:06:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:06:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:06:24.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:24.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:24.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:06:24.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:24.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:24.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:06:24.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:06:24.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 961 2024-08-25T20:06:24.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:25.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 960 2024-08-25T20:06:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:06:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:06:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:06:25.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:25.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:06:25.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:25.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:25.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:06:25.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:06:25.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:25.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 963 2024-08-25T20:06:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:26.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 962 2024-08-25T20:06:26.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:26.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:26.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:06:26.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:26.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:26.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:06:26.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:06:26.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 966 2024-08-25T20:06:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:06:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:06:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:06:26.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:27.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 964 2024-08-25T20:06:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:06:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47255 2024-08-25T20:06:27.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:27.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 967 2024-08-25T20:06:27.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:27.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:06:27.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:27.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:27.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:06:27.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:06:27.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:28.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 965 2024-08-25T20:06:28.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:28.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:06:28.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47387 2024-08-25T20:06:28.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:28.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:06:28.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:28.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:28.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:06:28.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:06:29.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:06:29.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47519 2024-08-25T20:06:29.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 968 2024-08-25T20:06:29.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:29.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:29.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:06:29.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:29.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:29.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:06:29.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:06:30.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 969 2024-08-25T20:06:30.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:06:30.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47671 2024-08-25T20:06:30.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:30.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:06:30.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:06:30.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:06:30.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:06:30.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:06:30.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:06:31.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 970 2024-08-25T20:06:31.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:31.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:06:31.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47824 2024-08-25T20:06:32.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 971 2024-08-25T20:06:32.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:06:32.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47977 2024-08-25T20:06:32.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:33.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 972 2024-08-25T20:06:33.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:06:33.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:06:33.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 48130 2024-08-25T20:06:34.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:06:34.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 48303 2024-08-25T20:06:35.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:06:35.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 48477 2024-08-25T20:06:36.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:06:36.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:06:36.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47124 2024-08-25T20:09:01.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:09:01.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:09:01.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:09:01.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47256 2024-08-25T20:09:02.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:09:02.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:09:02.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:09:02.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47388 2024-08-25T20:09:04.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:09:04.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:09:04.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:09:04.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47520 2024-08-25T20:09:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:09:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:09:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:09:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47672 2024-08-25T20:09:05.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:09:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:09:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:09:05.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47825 2024-08-25T20:09:06.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:09:06.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:09:06.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:09:06.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 47978 2024-08-25T20:09:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:09:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:09:07.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:09:07.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 48131 2024-08-25T20:09:08.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:09:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:09:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:09:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 48304 2024-08-25T20:09:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:09:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:09:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:09:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 48478 2024-08-25T20:09:10.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:09:10.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:09:10.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:09:10.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:09:10.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:09:10.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:09:10.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 1 2024-08-25T20:09:10.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:09:10.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd0 2024-08-25T20:09:10.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:09:10.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary1 2024-08-25T20:09:10.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary1 2024-08-25T20:09:10.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:09:10.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary1 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 2 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd1 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary2 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary2 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary2 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 3 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd2 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary3 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary3 2024-08-25T20:09:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary3 2024-08-25T20:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 4 2024-08-25T20:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd3 2024-08-25T20:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary4 2024-08-25T20:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary4 2024-08-25T20:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary4 2024-08-25T20:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:09:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 5 2024-08-25T20:09:10.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd4 2024-08-25T20:09:10.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary5 2024-08-25T20:09:10.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary5 2024-08-25T20:09:10.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:09:10.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary5 2024-08-25T20:09:10.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:09:10.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:09:10.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 6 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd5 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary6 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary6 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary6 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 7 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd6 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary7 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary7 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary7 2024-08-25T20:09:10.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:09:10.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 8 2024-08-25T20:09:10.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:09:10.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd7 2024-08-25T20:09:10.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary8 2024-08-25T20:09:10.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary8 2024-08-25T20:09:10.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:09:10.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary8 2024-08-25T20:09:10.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:09:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:09:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 9 2024-08-25T20:09:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:09:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:09:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 54431 2024-08-25T20:09:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd8 2024-08-25T20:09:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary9 2024-08-25T20:09:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary9 2024-08-25T20:09:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:09:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary9 2024-08-25T20:09:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 10 2024-08-25T20:09:10.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd9 2024-08-25T20:09:10.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary10 2024-08-25T20:09:10.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary10 2024-08-25T20:09:10.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:09:10.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary10 2024-08-25T20:09:11.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:09:11.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:09:11.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:09:11.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:09:11.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:09:11.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:09:11.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:09:11.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:09:11.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:09:11.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:09:11.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:09:11.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:09:11.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:09:11.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:09:11.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:09:11.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:09:11.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:09:11.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:09:11.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:09:11.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:09:59.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=4ad49366f9019695b7999aff17022b35 2024-08-25T20:09:59.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-08-25T20:09:59.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary9 2024-08-25T20:09:59.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:09:59.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:09:59.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:09:59.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-08-25T20:10:00.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:10:00.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-08-25T20:10:00.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:00.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:00.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:10:00.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:00.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:00.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:00.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:00.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:01.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:10:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:10:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:01.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:01.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:10:01.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:10:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eb517fd-e079-4cdb-85ca-27fc4caa8722 2024-08-25T20:10:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:01.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:01.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:01.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:01.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:10:03.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:10:03.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:03.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:03.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:10:03.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:03.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:03.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:03.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:03.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:03.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:10:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:10:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eb517fd-e079-4cdb-85ca-27fc4caa8722 2024-08-25T20:10:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:03.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:03.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:03.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:10:05.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=da30ecf4f49783bd21b8002f025bfcac 2024-08-25T20:10:05.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-08-25T20:10:06.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary3 2024-08-25T20:10:06.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:06.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:06.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:10:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-08-25T20:10:07.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=4a6daa93388f7d1431fc6785fa68160f 2024-08-25T20:10:07.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-08-25T20:10:07.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=31f3c2b134a8a49f45fa2ea04fc87845 2024-08-25T20:10:07.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-08-25T20:10:07.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=1b5700c43c90eac5e62fc57e80c531dd 2024-08-25T20:10:07.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-08-25T20:10:07.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary6 2024-08-25T20:10:07.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:07.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:07.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:10:07.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-08-25T20:10:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:10:07.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-08-25T20:10:07.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:07.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:07.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:10:07.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:07.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:07.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:07.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:07.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:07.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary10 2024-08-25T20:10:07.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:07.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:07.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:10:07.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-08-25T20:10:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary4 2024-08-25T20:10:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:10:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-08-25T20:10:07.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:10:07.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:07.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:07.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:10:07.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:07.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:07.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:07.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:07.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:07.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:10:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:10:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eb517fd-e079-4cdb-85ca-27fc4caa8722 2024-08-25T20:10:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:07.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:07.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:07.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=22f1aa51409deedecfc3b0a0671a23b7 2024-08-25T20:10:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-08-25T20:10:08.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=6ceb9ac1d3005a38845e03c043f94805 2024-08-25T20:10:08.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-08-25T20:10:08.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=defe79779c27b8653fdab7a9b623df7a 2024-08-25T20:10:08.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-08-25T20:10:08.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary1 2024-08-25T20:10:08.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:08.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:08.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:10:08.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-08-25T20:10:08.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary5 2024-08-25T20:10:08.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:08.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:08.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:10:08.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-08-25T20:10:08.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary2 2024-08-25T20:10:08.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:08.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:08.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:10:08.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-08-25T20:10:08.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:10:08.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:10:08.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:10:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:10:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-08-25T20:10:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:10:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-08-25T20:10:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:08.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:08.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:10:08.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:08.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:08.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:08.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:08.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-08-25T20:10:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:10:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:08.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-08-25T20:10:08.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:08.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:08.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:10:08.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:08.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:08.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:08.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:08.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:10:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:10:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:08.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:08.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:10:08.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=6643e3c15cd61a8af9610c9ee8ec4b2c 2024-08-25T20:10:08.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-08-25T20:10:08.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary8 2024-08-25T20:10:08.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:08.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:08.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:10:08.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-08-25T20:10:08.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:10:08.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 96ee44c8-c5eb-4e4b-9f36-cd4c135c2610 2024-08-25T20:10:08.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:08.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:08.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:08.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:08.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:08.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:08.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:08.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:10:08.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=de7bda15e92fc2760fa2e00d2cd3a442 2024-08-25T20:10:08.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-08-25T20:10:08.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary7 2024-08-25T20:10:08.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:08.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:08.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:10:08.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-08-25T20:10:09.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:10:09.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:10:09.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:10:09.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:10:09.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-08-25T20:10:09.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:09.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:09.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:09.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:09.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:09.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-08-25T20:10:09.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:09.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:09.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:10:09.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:09.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:09.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:09.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:09.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:09.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-08-25T20:10:09.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:09.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:09.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:10:09.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:09.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:09.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:09.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:09.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:09.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:10:09.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:09.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:09.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:10:09.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:09.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:09.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:09.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:10:09.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:09.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:09.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:09.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:09.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:09.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:10:09.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:09.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:10:09.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:09.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:10:09.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:10:09.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:09.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:09.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:10:09.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:09.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:09.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:09.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:09.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:09.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:10:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:10:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6113c176-30b4-41f0-a3f8-995119a61317 2024-08-25T20:10:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:09.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:09.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:09.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:09.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:10:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:10:09.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7dccc3b0-bd7c-48cd-afae-c170c8dcb807 2024-08-25T20:10:09.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:09.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:09.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:09.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:09.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:09.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:10:09.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: acfcd367-56d4-41f2-9237-93f74e314971 2024-08-25T20:10:09.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:09.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:09.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:09.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:09.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 40fe2c69-4e1c-45c2-8f78-7644b7280ef4 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:10:09.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:09.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:09.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:10:10.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:10:10.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:10:10.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:10.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:10.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:10.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:10.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:10:10.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:10.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:10.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:10.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:10.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:10.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:10:10.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:10.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:10:10.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:10:10.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e04d9be2-9a9b-45f2-aa7b-58edbddae93a 2024-08-25T20:10:10.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:10.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:10.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:10.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:10.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:10.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:10:10.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b74e74d2-ddac-461e-8089-2e96311f9b8c 2024-08-25T20:10:10.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:10.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:10.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:10.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:10.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:10.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:10:10.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dfe57283-5a78-4b59-8ca5-7585d4556e53 2024-08-25T20:10:10.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:10.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:10.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:10.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:10.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:10.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:10:10.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ec274d6d-37cd-4bf4-995b-bcac0ad1c987 2024-08-25T20:10:10.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:10.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:10.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:10.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:10.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:10.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:10.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:10.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:10:10.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:10.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:10.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:10:10.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:10.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:10.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:10:10.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:10.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:10.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:10:10.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:10:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:10:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:10:10.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:10:10.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 96ee44c8-c5eb-4e4b-9f36-cd4c135c2610 2024-08-25T20:10:10.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:10.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:10.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:10.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:10.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:10.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:10.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:10.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:10:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:10:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:10:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:11.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:11.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:10:11.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:10:11.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:11.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:11.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:10:11.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:11.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:11.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:11.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:11.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:11.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:10:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:10:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:10:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:11.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:11.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:10:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:10:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:10:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:10:11.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:10:11.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6113c176-30b4-41f0-a3f8-995119a61317 2024-08-25T20:10:11.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:11.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:11.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:11.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:11.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:10:11.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:10:11.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7dccc3b0-bd7c-48cd-afae-c170c8dcb807 2024-08-25T20:10:11.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:11.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:11.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:11.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:11.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:11.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:11.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:11.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:10:11.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:10:11.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: acfcd367-56d4-41f2-9237-93f74e314971 2024-08-25T20:10:11.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:11.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:11.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:11.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:11.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:11.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:11.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:11.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:10:11.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:10:11.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 40fe2c69-4e1c-45c2-8f78-7644b7280ef4 2024-08-25T20:10:11.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:11.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:11.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:11.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:11.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:11.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:11.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:11.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:10:12.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:10:12.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:12.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:12.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:10:12.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:12.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:12.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:12.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:12.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:12.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:10:12.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:10:12.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:12.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:10:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ec274d6d-37cd-4bf4-995b-bcac0ad1c987 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b74e74d2-ddac-461e-8089-2e96311f9b8c 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:12.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:10:12.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dfe57283-5a78-4b59-8ca5-7585d4556e53 2024-08-25T20:10:12.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:12.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:12.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:12.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:12.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:12.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:10:12.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e04d9be2-9a9b-45f2-aa7b-58edbddae93a 2024-08-25T20:10:12.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:12.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:12.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:12.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:12.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:12.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:12.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:12.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:10:12.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:12.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:12.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:10:12.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:12.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:12.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:10:12.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:12.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:12.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:10:14.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:10:14.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:14.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:14.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:10:14.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:14.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:14.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:14.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:14.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:14.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:10:14.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:10:14.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 96ee44c8-c5eb-4e4b-9f36-cd4c135c2610 2024-08-25T20:10:14.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:14.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:14.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:14.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:14.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:14.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:14.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:14.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:15.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:10:15.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:15.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:15.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:10:15.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:15.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:15.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:15.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:15.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:15.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:10:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:10:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:10:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:15.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:10:15.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:10:15.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:15.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:15.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:10:15.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:15.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:15.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:15.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:15.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:15.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:10:15.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:10:15.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:15.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:15.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:10:15.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:15.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:15.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:15.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:15.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:15.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:10:15.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:10:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6113c176-30b4-41f0-a3f8-995119a61317 2024-08-25T20:10:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:15.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:15.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:15.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:15.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:10:15.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7dccc3b0-bd7c-48cd-afae-c170c8dcb807 2024-08-25T20:10:15.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:15.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:15.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:15.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:15.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:15.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:10:15.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: acfcd367-56d4-41f2-9237-93f74e314971 2024-08-25T20:10:15.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:15.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:15.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:15.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:15.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:15.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:15.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:15.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:15.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:15.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:15.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 40fe2c69-4e1c-45c2-8f78-7644b7280ef4 2024-08-25T20:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:15.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:15.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:15.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:15.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:10:15.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:15.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:15.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:10:15.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:15.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:15.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:15.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:15.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:15.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:10:16.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:10:16.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eb517fd-e079-4cdb-85ca-27fc4caa8722 2024-08-25T20:10:16.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:16.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:16.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:16.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:16.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:16.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:16.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:16.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:16.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:10:16.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:16.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:16.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:10:16.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:16.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:16.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:16.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:16.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:16.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:16.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:16.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:10:16.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:16.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:10:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:10:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:10:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:10:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:16.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:10:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:10:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b74e74d2-ddac-461e-8089-2e96311f9b8c 2024-08-25T20:10:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:16.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:10:16.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ec274d6d-37cd-4bf4-995b-bcac0ad1c987 2024-08-25T20:10:16.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:16.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:16.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:16.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:16.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:16.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:10:16.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e04d9be2-9a9b-45f2-aa7b-58edbddae93a 2024-08-25T20:10:16.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:16.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:16.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:16.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:10:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dfe57283-5a78-4b59-8ca5-7585d4556e53 2024-08-25T20:10:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:16.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:16.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:16.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:16.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:16.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:16.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:16.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:16.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:22.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:10:22.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:22.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:22.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:10:22.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:22.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:22.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:22.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:22.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:22.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:10:22.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:10:22.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 96ee44c8-c5eb-4e4b-9f36-cd4c135c2610 2024-08-25T20:10:22.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:22.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:22.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:22.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:22.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:22.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:22.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:22.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:23.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:10:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:10:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:10:23.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:10:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:10:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:23.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:23.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:10:23.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:10:23.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:23.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:23.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:10:23.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:23.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:23.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:23.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:23.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:23.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:10:23.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:10:23.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:10:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:10:23.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:10:23.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6113c176-30b4-41f0-a3f8-995119a61317 2024-08-25T20:10:23.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:23.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:23.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:23.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:23.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:23.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:23.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:23.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:23.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:10:23.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: acfcd367-56d4-41f2-9237-93f74e314971 2024-08-25T20:10:23.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:23.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:23.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:23.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:23.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:23.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:10:23.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7dccc3b0-bd7c-48cd-afae-c170c8dcb807 2024-08-25T20:10:23.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:23.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:23.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:23.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:23.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:10:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 40fe2c69-4e1c-45c2-8f78-7644b7280ef4 2024-08-25T20:10:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:23.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:23.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:23.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:23.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:23.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:23.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:23.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:10:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:10:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:24.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:24.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:24.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:10:24.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:10:24.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eb517fd-e079-4cdb-85ca-27fc4caa8722 2024-08-25T20:10:24.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:24.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:24.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:24.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:24.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:24.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:24.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:24.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:24.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:10:24.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:24.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:24.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:10:24.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:24.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:24.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:24.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:24.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:24.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:10:24.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:10:24.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:24.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:24.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:10:24.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:24.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:24.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:24.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:24.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:24.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:10:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:10:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:10:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:10:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:10:24.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:24.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:24.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:24.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:24.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:24.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:10:24.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:24.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e04d9be2-9a9b-45f2-aa7b-58edbddae93a 2024-08-25T20:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ec274d6d-37cd-4bf4-995b-bcac0ad1c987 2024-08-25T20:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b74e74d2-ddac-461e-8089-2e96311f9b8c 2024-08-25T20:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:24.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:10:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dfe57283-5a78-4b59-8ca5-7585d4556e53 2024-08-25T20:10:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:10:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:10:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:09:58' 2024-08-25T20:10:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:24.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:10:24.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:24.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:10:30.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:10:30.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:30.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:30.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:10:30.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:30.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:30.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:30.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:30.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:30.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:10:30.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:10:30.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 96ee44c8-c5eb-4e4b-9f36-cd4c135c2610 2024-08-25T20:10:30.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:30.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:30.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:30.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:30.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:30.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:30.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:30.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:30.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:30.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:30.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:30.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:30.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:30.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:30.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:30.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:10:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:30.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:30.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:30.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:10:31.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:31.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:10:31.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:31.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:31.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:31.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:31.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:31.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:31.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-08-25T20:10:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:10:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:31.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:10:31.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:31.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:31.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:10:31.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:31.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:31.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:31.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:31.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:31.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:10:31.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:10:31.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:31.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:31.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:31.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:10:31.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:31.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:10:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:10:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:10:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:31.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:10:31.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:10:31.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6113c176-30b4-41f0-a3f8-995119a61317 2024-08-25T20:10:31.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:31.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:31.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:31.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:31.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:31.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:31.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:31.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:31.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:31.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:31.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:31.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:31.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:31.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:31.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:31.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:31.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:10:31.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: acfcd367-56d4-41f2-9237-93f74e314971 2024-08-25T20:10:31.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:31.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:31.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:31.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:31.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:10:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7dccc3b0-bd7c-48cd-afae-c170c8dcb807 2024-08-25T20:10:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:10:31.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:31.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:31.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:31.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:31.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:31.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:31.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:31.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:31.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:31.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:31.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:31.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:31.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:31.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:31.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:31.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:31.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:31.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:31.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:31.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:31.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:31.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:31.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:31.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:31.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:31.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:31.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:31.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:31.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:31.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:31.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:31.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:10:31.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:31.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:31.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:10:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:31.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:10:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 40fe2c69-4e1c-45c2-8f78-7644b7280ef4 2024-08-25T20:10:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:31.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:31.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:31.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:31.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:31.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:31.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:31.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:31.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:31.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:31.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:31.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:31.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:31.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:31.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:10:31.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:31.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:31.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:31.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:31.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:31.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:31.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:31.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:10:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:10:31.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:31.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:31.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:31.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-08-25T20:10:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:10:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:31.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:31.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:31.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:31.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:31.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:31.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:31.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:10:31.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:31.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:31.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:10:31.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:31.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:10:31.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:31.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:10:31.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:32.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:32.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:32.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-08-25T20:10:32.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:32.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:10:32.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:32.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:32.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-08-25T20:10:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:10:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:32.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:32.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:32.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:10:32.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:10:32.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-08-25T20:10:32.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:10:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-08-25T20:10:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:32.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:32.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-08-25T20:10:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:10:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:32.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:10:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:10:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:10:32.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:10:32.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 96ee44c8-c5eb-4e4b-9f36-cd4c135c2610 2024-08-25T20:10:32.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:32.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:32.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:32.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:32.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:32.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:32.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:32.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:32.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:32.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:32.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:32.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:32.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:32.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:32.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:32.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:32.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:10:32.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:32.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:32.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:10:32.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eb517fd-e079-4cdb-85ca-27fc4caa8722 2024-08-25T20:10:32.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:32.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:32.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:32.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:32.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:32.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:32.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:32.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:32.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:32.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:32.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:32.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:32.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:32.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:32.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:32.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:32.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:10:32.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:32.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:32.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:32.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:32.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:10:32.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:10:32.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:32.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:32.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary3 2024-08-25T20:10:32.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:32.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:10:32.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:10:32.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-08-25T20:10:32.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:32.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:32.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:10:32.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:10:32.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:32.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:32.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-08-25T20:10:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:10:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:10:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:10:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:10:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:10:32.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:10:32.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:10:32.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:10:32.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:32.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:10:32.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:10:32.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dfe57283-5a78-4b59-8ca5-7585d4556e53 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b74e74d2-ddac-461e-8089-2e96311f9b8c 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:10:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e04d9be2-9a9b-45f2-aa7b-58edbddae93a 2024-08-25T20:10:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:10:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ec274d6d-37cd-4bf4-995b-bcac0ad1c987 2024-08-25T20:10:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:32.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:32.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:32.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:32.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:32.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:32.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:32.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:32.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:32.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:32.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:32.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:32.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:32.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:32.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:32.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:32.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:32.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:32.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:32.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:32.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:32.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:32.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:32.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:32.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:32.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:32.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:32.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:32.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:10:32.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:10:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:10:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:10:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:10:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:32.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:32.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:32.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:10:32.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:10:32.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:32.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:32.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-08-25T20:10:32.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:32.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:10:32.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:32.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:32.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:32.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:32.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:32.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:32.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:10:32.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:10:32.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:32.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:32.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:10:32.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:32.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:10:32.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:10:32.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:10:32.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:32.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:32.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:32.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:32.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:32.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:32.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:32.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:32.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-08-25T20:10:32.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:32.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:10:32.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:32.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:32.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-08-25T20:10:32.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:32.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:10:32.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:10:32.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:10:32.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-08-25T20:10:32.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:32.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:32.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:10:32.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:32.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:32.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:32.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:32.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:32.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-08-25T20:10:32.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:10:32.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:10:32.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:10:32.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-08-25T20:10:33.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-08-25T20:10:33.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:10:33.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:33.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:33.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-08-25T20:10:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:10:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-08-25T20:10:33.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-08-25T20:10:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-08-25T20:10:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:10:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:33.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:33.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-08-25T20:10:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:10:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6113c176-30b4-41f0-a3f8-995119a61317 2024-08-25T20:10:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:33.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:33.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:33.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:33.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:33.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:33.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:33.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:33.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:33.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:33.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:33.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:33.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:10:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:33.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:10:33.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7dccc3b0-bd7c-48cd-afae-c170c8dcb807 2024-08-25T20:10:33.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:33.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:33.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:33.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:33.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:33.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:33.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:33.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:33.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:33.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:33.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:33.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:10:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: acfcd367-56d4-41f2-9237-93f74e314971 2024-08-25T20:10:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:33.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:33.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:33.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:33.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:33.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:10:33.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:33.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:33.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:33.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:33.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:33.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:33.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:33.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:33.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:10:33.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 40fe2c69-4e1c-45c2-8f78-7644b7280ef4 2024-08-25T20:10:33.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:33.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:33.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:33.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:33.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:33.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:33.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:33.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:33.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:33.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:33.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:33.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:10:33.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:33.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:33.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:33.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:33.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:33.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:33.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:33.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:33.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:10:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:33.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:33.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd0 2024-08-25T20:10:33.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:10:33.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:10:33.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:33.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:33.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:10:33.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:33.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:10:33.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:33.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:33.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:33.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary4 2024-08-25T20:10:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:10:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:10:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-08-25T20:10:33.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-08-25T20:10:33.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:10:33.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:33.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:33.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-08-25T20:10:33.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:33.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:10:33.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:33.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:10:33.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:33.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:10:33.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:33.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:33.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:33.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary10 2024-08-25T20:10:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:10:33.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:10:33.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-08-25T20:10:33.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:33.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:10:33.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:33.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:33.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:33.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:33.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:33.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary6 2024-08-25T20:10:33.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:10:33.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:10:33.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-08-25T20:10:33.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:10:33.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:33.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:10:33.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:33.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:33.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:33.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:33.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary1 2024-08-25T20:10:33.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:10:33.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:10:33.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-08-25T20:10:33.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:10:33.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eb517fd-e079-4cdb-85ca-27fc4caa8722 2024-08-25T20:10:33.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:33.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:33.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:33.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:33.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:33.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:33.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:33.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:33.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:33.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:33.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:33.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:33.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:33.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:33.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:33.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:33.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:33.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:10:33.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:33.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:33.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:33.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:33.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:10:33.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:33.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:10:33.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:33.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:33.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:33.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:33.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:33.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:33.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary9 2024-08-25T20:10:33.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:10:33.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:10:33.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-08-25T20:10:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-08-25T20:10:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:10:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-08-25T20:10:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-08-25T20:10:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:10:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-08-25T20:10:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:10:33.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:33.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-08-25T20:10:33.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-08-25T20:10:33.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:33.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-08-25T20:10:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:10:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e04d9be2-9a9b-45f2-aa7b-58edbddae93a 2024-08-25T20:10:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:33.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:33.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:34.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:34.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:34.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:34.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:34.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:34.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:34.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:34.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:10:34.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:10:34.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:10:34.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:10:34.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:10:34.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:34.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:34.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:34.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:34.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:10:34.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:34.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:34.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:34.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:10:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b74e74d2-ddac-461e-8089-2e96311f9b8c 2024-08-25T20:10:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:34.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:34.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:34.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:34.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:34.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:10:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dfe57283-5a78-4b59-8ca5-7585d4556e53 2024-08-25T20:10:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:34.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:34.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:34.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:10:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ec274d6d-37cd-4bf4-995b-bcac0ad1c987 2024-08-25T20:10:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:10:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:10:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:10:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:10:28' 2024-08-25T20:10:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:10:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:10:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:10:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:10:34.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:10:34.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:10:34.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:10:34.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:34.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-08-25T20:10:34.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:34.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:34.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:34.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:34.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:34.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-08-25T20:10:34.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:10:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:10:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:10:34.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:10:34.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:10:34.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:10:34.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:34.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-08-25T20:10:34.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-08-25T20:10:34.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:34.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:10:34.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:10:34.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:10:34.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:34.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:34.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-08-25T20:10:34.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:10:34.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:34.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:34.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:34.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:34.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:34.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:34.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:34.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:10:34.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:34.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:34.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:34.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:10:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:34.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:34.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:10:34.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:34.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:34.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:34.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:10:34.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:34.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:10:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-08-25 20:10:28' 2024-08-25T20:10:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:10:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:10:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:10:28' 2024-08-25T20:10:34.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:10:34.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:10:34.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:10:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:34.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:34.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:10:34.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:34.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:10:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:34.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:34.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:34.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary2 2024-08-25T20:10:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:10:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:10:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-08-25T20:10:34.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd1 2024-08-25T20:10:34.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd2 2024-08-25T20:10:34.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:10:34.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:10:34.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:10:34.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:10:34.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:34.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:34.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:10:34.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:34.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:10:34.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:34.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:10:34.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:10:28' 2024-08-25T20:10:34.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:10:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:10:34.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:34.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:34.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:10:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:34.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:34.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:34.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary7 2024-08-25T20:10:34.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:10:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:10:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-08-25T20:10:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:10:28' 2024-08-25T20:10:34.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:10:34.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd3 2024-08-25T20:10:34.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:34.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd5 2024-08-25T20:10:34.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:34.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:10:34.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:10:34.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:34.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:10:34.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:34.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:10:34.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:10:34.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:10:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:10:34.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:34.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:34.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary8 2024-08-25T20:10:34.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:34.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:34.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:10:34.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:10:34.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-08-25T20:10:34.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:10:34.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:34.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:10:34.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary5 2024-08-25T20:10:34.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:10:34.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:10:34.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:10:34.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:10:34.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-08-25T20:10:34.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd4 2024-08-25T20:10:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:10:34.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:10:35.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:10:35.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:10:35.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:10:35.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:10:35.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:10:35.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-08-25T20:10:35.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:10:35.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:10:35.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:10:35.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:10:35.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:10:35.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-08-25T20:10:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-08-25T20:10:35.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:10:35.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:10:35.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-08-25T20:10:35.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd6 2024-08-25T20:10:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd7 2024-08-25T20:10:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:10:35.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:10:35.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:10:35.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:10:35.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd8 2024-08-25T20:10:35.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:10:35.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:10:35.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd9 2024-08-25T20:10:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:10:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:11:28.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=da30ecf4f49783bd21b8002f025bfcac 2024-08-25T20:11:28.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-08-25T20:11:28.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ da30ecf4f49783bd21b8002f025bfcac != \d\a\3\0\e\c\f\4\f\4\9\7\8\3\b\d\2\1\b\8\0\0\2\f\0\2\5\b\f\c\a\c ]] 2024-08-25T20:11:28.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=4a6daa93388f7d1431fc6785fa68160f 2024-08-25T20:11:28.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-08-25T20:11:29.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 4a6daa93388f7d1431fc6785fa68160f != \4\a\6\d\a\a\9\3\3\8\8\f\7\d\1\4\3\1\f\c\6\7\8\5\f\a\6\8\1\6\0\f ]] 2024-08-25T20:11:29.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=6ceb9ac1d3005a38845e03c043f94805 2024-08-25T20:11:29.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-08-25T20:11:29.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=defe79779c27b8653fdab7a9b623df7a 2024-08-25T20:11:29.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-08-25T20:11:29.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 6ceb9ac1d3005a38845e03c043f94805 != \6\c\e\b\9\a\c\1\d\3\0\0\5\a\3\8\8\4\5\e\0\3\c\0\4\3\f\9\4\8\0\5 ]] 2024-08-25T20:11:29.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ defe79779c27b8653fdab7a9b623df7a != \d\e\f\e\7\9\7\7\9\c\2\7\b\8\6\5\3\f\d\a\b\7\a\9\b\6\2\3\d\f\7\a ]] 2024-08-25T20:11:29.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=31f3c2b134a8a49f45fa2ea04fc87845 2024-08-25T20:11:29.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-08-25T20:11:29.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=22f1aa51409deedecfc3b0a0671a23b7 2024-08-25T20:11:29.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-08-25T20:11:29.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 31f3c2b134a8a49f45fa2ea04fc87845 != \3\1\f\3\c\2\b\1\3\4\a\8\a\4\9\f\4\5\f\a\2\e\a\0\4\f\c\8\7\8\4\5 ]] 2024-08-25T20:11:29.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 22f1aa51409deedecfc3b0a0671a23b7 != \2\2\f\1\a\a\5\1\4\0\9\d\e\e\d\e\c\f\c\3\b\0\a\0\6\7\1\a\2\3\b\7 ]] 2024-08-25T20:11:29.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:11:29.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 54432 2024-08-25T20:11:29.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:11:29.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 54433 2024-08-25T20:11:29.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:11:29.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 54436 2024-08-25T20:11:30.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=6643e3c15cd61a8af9610c9ee8ec4b2c 2024-08-25T20:11:30.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-08-25T20:11:30.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=1b5700c43c90eac5e62fc57e80c531dd 2024-08-25T20:11:30.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-08-25T20:11:30.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 6643e3c15cd61a8af9610c9ee8ec4b2c != \6\6\4\3\e\3\c\1\5\c\d\6\1\a\8\a\f\9\6\1\0\c\9\e\e\8\e\c\4\b\2\c ]] 2024-08-25T20:11:30.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 1b5700c43c90eac5e62fc57e80c531dd != \1\b\5\7\0\0\c\4\3\c\9\0\e\a\c\5\e\6\2\f\c\5\7\e\8\0\c\5\3\1\d\d ]] 2024-08-25T20:11:30.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:11:30.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 54438 2024-08-25T20:11:30.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:11:30.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 54440 2024-08-25T20:11:30.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:11:30.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 54442 2024-08-25T20:11:30.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=4ad49366f9019695b7999aff17022b35 2024-08-25T20:11:30.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-08-25T20:11:30.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 4ad49366f9019695b7999aff17022b35 != \4\a\d\4\9\3\6\6\f\9\0\1\9\6\9\5\b\7\9\9\9\a\f\f\1\7\0\2\2\b\3\5 ]] 2024-08-25T20:11:31.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=de7bda15e92fc2760fa2e00d2cd3a442 2024-08-25T20:11:31.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-08-25T20:11:31.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ de7bda15e92fc2760fa2e00d2cd3a442 != \d\e\7\b\d\a\1\5\e\9\2\f\c\2\7\6\0\f\a\2\e\0\0\d\2\c\d\3\a\4\4\2 ]] 2024-08-25T20:11:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:11:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 54444 2024-08-25T20:11:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:11:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 54446 2024-08-25T20:11:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:11:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 54448 2024-08-25T20:11:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:11:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:11:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:11:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-08-25T20:11:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:31.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:11:31.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:11:31.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-08-25T20:11:31.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:31.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:31.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:11:31.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:31.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:31.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:31.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:31.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:11:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-08-25T20:11:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:32.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:11:32.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:32.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:32.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:32.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:32.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:32.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:11:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:11:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 40fe2c69-4e1c-45c2-8f78-7644b7280ef4 2024-08-25T20:11:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:11:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3182387.2,"bytes_per_snapshot":63647744.0,"last_snapshot_bytes":63647744,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"} 2024-08-25T20:11:32.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:11:32.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:11:28' 2024-08-25T20:11:32.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:11:32.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:11:32.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:11:32.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3182387.2,"bytes_per_snapshot":63647744.0,"last_snapshot_bytes":63647744,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"} 2024-08-25T20:11:32.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:11:32.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:32.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:11:32.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:32.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:11:32.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:11:32.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:11:32.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:11:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-08-25 20:11:28' 2024-08-25T20:11:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:11:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:11:28' 2024-08-25T20:11:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:11:32.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary140fe2c69-4e1c-45c2-8f78-7644b7280ef4up+replayingreplaying, {"bytes_per_second":3182387.2,"bytes_per_snapshot":63647744.0,"last_snapshot_bytes":63647744,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+replayingreplaying, {"bytes_per_second":3186892.8,"bytes_per_snapshot":63737856.0,"last_snapshot_bytes":63737856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+replayingreplaying, {"bytes_per_second":3257753.6,"bytes_per_snapshot":65155072.0,"last_snapshot_bytes":65155072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+replayingreplaying, {"bytes_per_second":3302400.0,"bytes_per_snapshot":66048000.0,"last_snapshot_bytes":66048000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616632,"remote_snapshot_timestamp":1724616632,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+replayingreplaying, {"bytes_per_second":3258572.8,"bytes_per_snapshot":65171456.0,"last_snapshot_bytes":65171456,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+replayingreplaying, {"bytes_per_second":3348889.6,"bytes_per_snapshot":66977792.0,"last_snapshot_bytes":66977792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+replayingreplaying, {"bytes_per_second":3191398.4,"bytes_per_snapshot":63827968.0,"last_snapshot_bytes":63827968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+replayingreplaying, {"bytes_per_second":3179520.0,"bytes_per_snapshot":63590400.0,"last_snapshot_bytes":63590400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28' 2024-08-25T20:11:32.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:11:32.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:11:32.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:32.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:11:32.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:11:32.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:32.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:11:32.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:32.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:32.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:32.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary1 2024-08-25T20:11:32.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:11:32.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:32.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:11:32.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-08-25T20:11:32.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-08-25T20:11:33.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:33.093+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3388340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:11:33.099 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:33.093+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ae4fe2a00 [3/40fe2c69-4e1c-45c2-8f78-7644b7280ef4] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:11:34.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:11:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-08-25T20:11:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:11:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:11:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary1 2024-08-25T20:11:34.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:34.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary1 2024-08-25T20:11:34.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:11:34.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:11:34.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-08-25T20:11:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:11:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:11:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-08-25T20:11:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:11:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:34.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:11:35.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-08-25T20:11:35.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:35.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:35.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:11:35.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:35.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:35.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:35.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:35.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:35.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:11:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:11:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e04d9be2-9a9b-45f2-aa7b-58edbddae93a 2024-08-25T20:11:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:11:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3257753.6,"bytes_per_snapshot":65155072.0,"last_snapshot_bytes":65155072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"} 2024-08-25T20:11:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:11:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:11:28' 2024-08-25T20:11:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:11:35.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:11:35.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:11:35.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3257753.6,"bytes_per_snapshot":65155072.0,"last_snapshot_bytes":65155072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"} 2024-08-25T20:11:35.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:11:35.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:11:35.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:35.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:11:35.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:11:35.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:11:35.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:11:35.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:35.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-08-25 20:11:28' 2024-08-25T20:11:35.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:35.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:35.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:11:35.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:35.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:11:28' 2024-08-25T20:11:35.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:35.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:35.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:11:35.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING941904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+replayingreplaying, {"bytes_per_second":3186892.8,"bytes_per_snapshot":63737856.0,"last_snapshot_bytes":63737856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary2e04d9be2-9a9b-45f2-aa7b-58edbddae93aup+replayingreplaying, {"bytes_per_second":3257753.6,"bytes_per_snapshot":65155072.0,"last_snapshot_bytes":65155072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+replayingreplaying, {"bytes_per_second":3302400.0,"bytes_per_snapshot":66048000.0,"last_snapshot_bytes":66048000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616632,"remote_snapshot_timestamp":1724616632,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+replayingreplaying, {"bytes_per_second":3258572.8,"bytes_per_snapshot":65171456.0,"last_snapshot_bytes":65171456,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+replayingreplaying, {"bytes_per_second":3348889.6,"bytes_per_snapshot":66977792.0,"last_snapshot_bytes":66977792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+replayingreplaying, {"bytes_per_second":3191398.4,"bytes_per_snapshot":63827968.0,"last_snapshot_bytes":63827968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+replayingreplaying, {"bytes_per_second":3179520.0,"bytes_per_snapshot":63590400.0,"last_snapshot_bytes":63590400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28' 2024-08-25T20:11:35.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:11:35.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:11:35.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:35.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:11:35.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:11:35.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:35.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:11:35.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary2 2024-08-25T20:11:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:11:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:11:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-08-25T20:11:35.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-08-25T20:11:35.709 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:35.708+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4e69860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:11:35.709 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:35.708+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972280 [3/e04d9be2-9a9b-45f2-aa7b-58edbddae93a] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:11:37.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:11:37.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-08-25T20:11:37.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:37.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:37.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:11:37.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:11:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary2 2024-08-25T20:11:37.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:37.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary2 2024-08-25T20:11:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:11:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:11:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-08-25T20:11:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:11:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:11:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-08-25T20:11:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:11:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:11:38.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-08-25T20:11:38.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:11:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:11:38.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:11:38.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 96ee44c8-c5eb-4e4b-9f36-cd4c135c2610 2024-08-25T20:11:38.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:11:38.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3302400.0,"bytes_per_snapshot":66048000.0,"last_snapshot_bytes":66048000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616632,"remote_snapshot_timestamp":1724616632,"replay_state":"idle"} 2024-08-25T20:11:38.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:11:38.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:11:28' 2024-08-25T20:11:38.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:11:38.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:11:38.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:11:38.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3302400.0,"bytes_per_snapshot":66048000.0,"last_snapshot_bytes":66048000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616632,"remote_snapshot_timestamp":1724616632,"replay_state":"idle"} 2024-08-25T20:11:38.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:11:38.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:38.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:11:38.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:38.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:11:38.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:11:38.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:11:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:11:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-08-25 20:11:28' 2024-08-25T20:11:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:11:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:11:28' 2024-08-25T20:11:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:11:38.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING841904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+replayingreplaying, {"bytes_per_second":3186892.8,"bytes_per_snapshot":63737856.0,"last_snapshot_bytes":63737856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary396ee44c8-c5eb-4e4b-9f36-cd4c135c2610up+replayingreplaying, {"bytes_per_second":3302400.0,"bytes_per_snapshot":66048000.0,"last_snapshot_bytes":66048000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616632,"remote_snapshot_timestamp":1724616632,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+replayingreplaying, {"bytes_per_second":3258572.8,"bytes_per_snapshot":65171456.0,"last_snapshot_bytes":65171456,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+replayingreplaying, {"bytes_per_second":3348889.6,"bytes_per_snapshot":66977792.0,"last_snapshot_bytes":66977792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+replayingreplaying, {"bytes_per_second":3191398.4,"bytes_per_snapshot":63827968.0,"last_snapshot_bytes":63827968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+replayingreplaying, {"bytes_per_second":3179520.0,"bytes_per_snapshot":63590400.0,"last_snapshot_bytes":63590400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28' 2024-08-25T20:11:38.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:11:38.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:11:38.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:38.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:11:38.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:11:38.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:38.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:11:38.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:38.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:38.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:38.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary3 2024-08-25T20:11:38.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:11:38.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:38.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:11:38.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-08-25T20:11:38.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-08-25T20:11:38.770 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:38.769+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4c3a9c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:11:38.770 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:38.769+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972c80 [3/96ee44c8-c5eb-4e4b-9f36-cd4c135c2610] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:11:39.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:11:39.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-08-25T20:11:39.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:39.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:39.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:11:39.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:39.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:11:39.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary3 2024-08-25T20:11:39.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:39.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary3 2024-08-25T20:11:39.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:11:39.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:11:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-08-25T20:11:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:11:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:40.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:40.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:40.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:40.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:11:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:11:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6113c176-30b4-41f0-a3f8-995119a61317 2024-08-25T20:11:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:11:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3258572.8,"bytes_per_snapshot":65171456.0,"last_snapshot_bytes":65171456,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"} 2024-08-25T20:11:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:11:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:11:28' 2024-08-25T20:11:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:11:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:11:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:11:41.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3258572.8,"bytes_per_snapshot":65171456.0,"last_snapshot_bytes":65171456,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"} 2024-08-25T20:11:41.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:11:41.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:11:41.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:41.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:11:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:11:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:11:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:11:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-08-25 20:11:28' 2024-08-25T20:11:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:11:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:11:28' 2024-08-25T20:11:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:11:41.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING741904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+replayingreplaying, {"bytes_per_second":3186892.8,"bytes_per_snapshot":63737856.0,"last_snapshot_bytes":63737856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary46113c176-30b4-41f0-a3f8-995119a61317up+replayingreplaying, {"bytes_per_second":3258572.8,"bytes_per_snapshot":65171456.0,"last_snapshot_bytes":65171456,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+replayingreplaying, {"bytes_per_second":3348889.6,"bytes_per_snapshot":66977792.0,"last_snapshot_bytes":66977792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+replayingreplaying, {"bytes_per_second":3191398.4,"bytes_per_snapshot":63827968.0,"last_snapshot_bytes":63827968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+replayingreplaying, {"bytes_per_second":3179520.0,"bytes_per_snapshot":63590400.0,"last_snapshot_bytes":63590400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28' 2024-08-25T20:11:41.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:11:41.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:11:41.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:41.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:11:41.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:11:41.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:41.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:11:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary4 2024-08-25T20:11:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:11:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:11:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-08-25T20:11:41.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-08-25T20:11:41.328 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:41.327+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adf01cd00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:11:41.328 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:41.327+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555adf432500 [3/6113c176-30b4-41f0-a3f8-995119a61317] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:11:42.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:11:42.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-08-25T20:11:42.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:42.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:11:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:11:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary4 2024-08-25T20:11:42.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:42.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary4 2024-08-25T20:11:42.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:11:42.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:11:42.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:11:43.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-08-25T20:11:43.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:43.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:43.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:11:43.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:43.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:43.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:43.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:11:44.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:11:44.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dfe57283-5a78-4b59-8ca5-7585d4556e53 2024-08-25T20:11:44.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:11:44.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"} 2024-08-25T20:11:44.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:11:44.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:11:28' 2024-08-25T20:11:44.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:11:44.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:11:44.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:11:44.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"} 2024-08-25T20:11:44.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:11:44.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:44.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:11:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:11:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:11:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:11:44.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:11:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-08-25 20:11:28' 2024-08-25T20:11:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:11:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:11:28' 2024-08-25T20:11:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:11:44.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING641904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+replayingreplaying, {"bytes_per_second":3186892.8,"bytes_per_snapshot":63737856.0,"last_snapshot_bytes":63737856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary5dfe57283-5a78-4b59-8ca5-7585d4556e53up+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+replayingreplaying, {"bytes_per_second":3348889.6,"bytes_per_snapshot":66977792.0,"last_snapshot_bytes":66977792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+replayingreplaying, {"bytes_per_second":3191398.4,"bytes_per_snapshot":63827968.0,"last_snapshot_bytes":63827968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+replayingreplaying, {"bytes_per_second":3179520.0,"bytes_per_snapshot":63590400.0,"last_snapshot_bytes":63590400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28' 2024-08-25T20:11:44.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:11:44.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:11:44.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:44.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:11:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:11:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:11:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary5 2024-08-25T20:11:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:11:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:11:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-08-25T20:11:44.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-08-25T20:11:44.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:44.330+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adf4c81a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:11:44.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:44.330+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972000 [3/dfe57283-5a78-4b59-8ca5-7585d4556e53] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:11:45.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:11:45.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-08-25T20:11:45.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:45.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:45.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:11:45.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:45.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:11:45.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary5 2024-08-25T20:11:45.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:45.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary5 2024-08-25T20:11:45.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:45.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:45.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:11:46.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-08-25T20:11:46.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:46.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:46.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:11:46.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:46.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:46.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:46.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:46.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:46.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:11:47.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:11:47.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: acfcd367-56d4-41f2-9237-93f74e314971 2024-08-25T20:11:47.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:11:47.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3348889.6,"bytes_per_snapshot":66977792.0,"last_snapshot_bytes":66977792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"} 2024-08-25T20:11:47.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:11:47.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:11:28' 2024-08-25T20:11:47.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:11:47.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:11:47.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:11:47.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3348889.6,"bytes_per_snapshot":66977792.0,"last_snapshot_bytes":66977792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"} 2024-08-25T20:11:47.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:11:47.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:47.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:11:47.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:47.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:11:47.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:11:47.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:11:47.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:11:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-08-25 20:11:28' 2024-08-25T20:11:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:11:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:47.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:11:28' 2024-08-25T20:11:47.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:47.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:47.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:11:47.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING541904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+replayingreplaying, {"bytes_per_second":3186892.8,"bytes_per_snapshot":63737856.0,"last_snapshot_bytes":63737856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary6acfcd367-56d4-41f2-9237-93f74e314971up+replayingreplaying, {"bytes_per_second":3348889.6,"bytes_per_snapshot":66977792.0,"last_snapshot_bytes":66977792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+replayingreplaying, {"bytes_per_second":3191398.4,"bytes_per_snapshot":63827968.0,"last_snapshot_bytes":63827968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+replayingreplaying, {"bytes_per_second":3179520.0,"bytes_per_snapshot":63590400.0,"last_snapshot_bytes":63590400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28' 2024-08-25T20:11:47.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:11:47.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:11:47.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:47.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:11:47.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:11:47.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:47.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:11:47.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:47.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:47.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:47.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary6 2024-08-25T20:11:47.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:11:47.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:47.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:11:47.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-08-25T20:11:47.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-08-25T20:11:47.409 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:47.407+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4fcb860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:11:47.410 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:47.407+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972780 [3/acfcd367-56d4-41f2-9237-93f74e314971] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:11:48.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:11:48.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-08-25T20:11:48.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:48.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:48.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:11:48.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:48.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:11:48.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary6 2024-08-25T20:11:48.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:48.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary6 2024-08-25T20:11:48.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:11:48.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:11:48.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-08-25T20:11:48.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:48.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:48.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:11:48.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:11:48.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-08-25T20:11:48.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:48.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:48.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:11:48.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:48.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:48.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:48.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:48.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:11:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-08-25T20:11:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:11:49.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:49.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:49.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:49.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:49.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:49.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:11:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:11:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b74e74d2-ddac-461e-8089-2e96311f9b8c 2024-08-25T20:11:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:11:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"} 2024-08-25T20:11:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:11:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:11:28' 2024-08-25T20:11:50.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:11:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:11:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:11:50.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"} 2024-08-25T20:11:50.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:11:50.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:50.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:11:50.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:50.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:11:50.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:11:50.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:11:50.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:11:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-08-25 20:11:28' 2024-08-25T20:11:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:11:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:11:28' 2024-08-25T20:11:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:50.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:11:50.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING441904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+replayingreplaying, {"bytes_per_second":3186892.8,"bytes_per_snapshot":63737856.0,"last_snapshot_bytes":63737856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary7b74e74d2-ddac-461e-8089-2e96311f9b8cup+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+replayingreplaying, {"bytes_per_second":3191398.4,"bytes_per_snapshot":63827968.0,"last_snapshot_bytes":63827968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+replayingreplaying, {"bytes_per_second":3179520.0,"bytes_per_snapshot":63590400.0,"last_snapshot_bytes":63590400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28' 2024-08-25T20:11:50.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:11:50.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:11:50.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:50.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:11:50.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:11:50.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:50.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:11:50.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:50.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:50.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:50.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary7 2024-08-25T20:11:50.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:11:50.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:50.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:11:50.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-08-25T20:11:50.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-08-25T20:11:50.323 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:50.321+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8b1860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:11:50.323 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:50.321+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973b80 [3/b74e74d2-ddac-461e-8089-2e96311f9b8c] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:11:51.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:11:51.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-08-25T20:11:51.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:51.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:51.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:11:51.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:11:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary7 2024-08-25T20:11:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary7 2024-08-25T20:11:51.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:11:51.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:11:51.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-08-25T20:11:51.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:51.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:51.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:11:51.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:11:51.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-08-25T20:11:51.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:51.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:51.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:11:51.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:51.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:51.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:51.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:51.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:11:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-08-25T20:11:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:11:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:11:53.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:11:53.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ec274d6d-37cd-4bf4-995b-bcac0ad1c987 2024-08-25T20:11:53.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:11:53.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3191398.4,"bytes_per_snapshot":63827968.0,"last_snapshot_bytes":63827968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"} 2024-08-25T20:11:53.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:11:53.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:11:28' 2024-08-25T20:11:53.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:11:53.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:11:53.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:11:53.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3191398.4,"bytes_per_snapshot":63827968.0,"last_snapshot_bytes":63827968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"} 2024-08-25T20:11:53.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:11:53.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:53.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:11:53.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:53.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:11:53.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:11:53.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:11:53.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:11:53.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:53.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-08-25 20:11:28' 2024-08-25T20:11:53.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:53.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:53.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:11:53.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:53.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:11:28' 2024-08-25T20:11:53.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:53.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:53.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:11:53.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING341904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+replayingreplaying, {"bytes_per_second":3186892.8,"bytes_per_snapshot":63737856.0,"last_snapshot_bytes":63737856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary8ec274d6d-37cd-4bf4-995b-bcac0ad1c987up+replayingreplaying, {"bytes_per_second":3191398.4,"bytes_per_snapshot":63827968.0,"last_snapshot_bytes":63827968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616634,"remote_snapshot_timestamp":1724616634,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+replayingreplaying, {"bytes_per_second":3179520.0,"bytes_per_snapshot":63590400.0,"last_snapshot_bytes":63590400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28' 2024-08-25T20:11:53.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:11:53.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:11:53.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:53.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:11:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:11:53.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:53.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:11:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary8 2024-08-25T20:11:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:11:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:11:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-08-25T20:11:53.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-08-25T20:11:53.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:53.334+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae387f520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:11:53.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:53.334+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972500 [3/ec274d6d-37cd-4bf4-995b-bcac0ad1c987] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:11:54.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:11:54.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-08-25T20:11:54.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:54.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:54.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:11:54.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:54.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:11:54.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary8 2024-08-25T20:11:54.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:54.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary8 2024-08-25T20:11:54.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:11:54.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:11:54.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:11:55.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-08-25T20:11:55.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:55.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:55.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:11:55.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:55.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:55.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:55.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:55.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:55.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:11:56.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:11:56.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eb517fd-e079-4cdb-85ca-27fc4caa8722 2024-08-25T20:11:56.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:11:56.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3179520.0,"bytes_per_snapshot":63590400.0,"last_snapshot_bytes":63590400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"} 2024-08-25T20:11:56.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:11:56.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:11:28' 2024-08-25T20:11:56.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:11:56.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:11:56.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:11:56.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3179520.0,"bytes_per_snapshot":63590400.0,"last_snapshot_bytes":63590400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"} 2024-08-25T20:11:56.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:11:56.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:56.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:11:56.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:56.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:11:56.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:11:56.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:11:56.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:11:56.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:56.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-08-25 20:11:28' 2024-08-25T20:11:56.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:56.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:56.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:11:56.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:56.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:11:28' 2024-08-25T20:11:56.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:56.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:56.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:11:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING241904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+replayingreplaying, {"bytes_per_second":3186892.8,"bytes_per_snapshot":63737856.0,"last_snapshot_bytes":63737856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28image-primary95eb517fd-e079-4cdb-85ca-27fc4caa8722up+replayingreplaying, {"bytes_per_second":3179520.0,"bytes_per_snapshot":63590400.0,"last_snapshot_bytes":63590400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:28' 2024-08-25T20:11:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:11:56.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:11:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:28' 2024-08-25T20:11:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:11:56.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:11:56.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:56.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:11:56.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:56.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:56.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:56.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary9 2024-08-25T20:11:56.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:11:56.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:56.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:11:56.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-08-25T20:11:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-08-25T20:11:56.447 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:56.445+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adf823ba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:11:56.447 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:56.445+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973180 [3/5eb517fd-e079-4cdb-85ca-27fc4caa8722] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:11:58.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:11:58.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-08-25T20:11:58.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:58.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:58.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:11:58.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:58.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:11:58.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary9 2024-08-25T20:11:58.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:58.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary9 2024-08-25T20:11:58.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:11:58.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:11:58.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-08-25T20:11:58.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:58.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:58.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:11:58.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:11:58.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-08-25T20:11:58.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:58.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:58.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:11:58.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:58.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:58.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:58.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:11:58.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:11:59.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-08-25T20:11:59.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:59.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:59.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:11:59.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:59.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:11:59.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:11:59.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:59.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:59.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:11:59.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:11:59.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7dccc3b0-bd7c-48cd-afae-c170c8dcb807 2024-08-25T20:11:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:11:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63737856.0,"last_snapshot_bytes":63737856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"} 2024-08-25T20:11:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:11:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:11:58' 2024-08-25T20:11:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:11:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:11:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:11:59.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63737856.0,"last_snapshot_bytes":63737856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"} 2024-08-25T20:11:59.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:11:59.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:59.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:11:59.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:59.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:11:59.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:11:59.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:11:59.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:11:59.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:58' 2024-08-25T20:11:59.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-08-25 20:11:58' 2024-08-25T20:11:59.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:11:59.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:59.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:11:59.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:11:59.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:11:58' 2024-08-25T20:11:59.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:11:59.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:11:59.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:11:59.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING141904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary107dccc3b0-bd7c-48cd-afae-c170c8dcb807up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63737856.0,"last_snapshot_bytes":63737856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724616633,"remote_snapshot_timestamp":1724616633,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:11:58' 2024-08-25T20:11:59.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:11:59.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:11:59.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:11:58' 2024-08-25T20:11:59.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:11:59.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:11:59.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:11:59.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:11:59.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:11:59.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:59.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:11:59.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary10 2024-08-25T20:11:59.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:11:59.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:11:59.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:11:59.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-08-25T20:11:59.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-08-25T20:11:59.571 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:59.570+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4e69520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:11:59.571 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:11:59.570+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972f00 [3/7dccc3b0-bd7c-48cd-afae-c170c8dcb807] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:12:00.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:12:00.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-08-25T20:12:00.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:00.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:00.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:00.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:12:00.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:12:00.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary10 2024-08-25T20:12:00.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:12:00.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary10 2024-08-25T20:12:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..10} 2024-08-25T20:12:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS=() 2024-08-25T20:12:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS=() 2024-08-25T20:12:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS=() 2024-08-25T20:12:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS=() 2024-08-25T20:12:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RET=0 2024-08-25T20:12:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:12:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:12:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary1 2024-08-25T20:12:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-08-25T20:12:00.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:00.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:00.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:12:00.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:12:00.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:12:00.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:00.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-08-25T20:12:01.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:12:01.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:01.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:12:01.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:12:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-08-25T20:12:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-08-25T20:12:01.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:12:01.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-08-25T20:12:01.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary1': 2024-08-25T20:12:01.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:12:01.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:12:01.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:12:01.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1fb72a6383ae 2024-08-25T20:12:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1fb72a6383ae 2024-08-25T20:12:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:12:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:12:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:12:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:12:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:12:01 2024 2024-08-25T20:12:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:12:01 2024 2024-08-25T20:12:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:12:01 2024 2024-08-25T20:12:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:12:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:12:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 5bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4 2024-08-25T20:12:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:12:01.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:12:01.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:12:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-08-25T20:12:01.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd0 2024-08-25T20:12:01.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:12:01.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-08-25T20:12:01.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:12:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:12:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:12:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 7e0d6291-b31b-4494-a789-39cb87948da3 2024-08-25T20:12:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:12:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:12:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:12:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:12:01.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:12:01.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:12:01.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:01.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary1 2024-08-25T20:12:01.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-08-25T20:12:01.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:12:01.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary1 2024-08-25T20:12:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:12:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-08-25T20:12:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:12:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary2 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary1 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary1 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:12:01.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:12:01.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:12:01.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:01.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-08-25T20:12:01.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:12:01.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:01.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:12:01.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:12:01.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-08-25T20:12:01.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:01.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:01.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:01.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:01.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-08-25T20:12:01.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:12:01.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-08-25T20:12:02.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:12:02.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 994 2024-08-25T20:12:02.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:02.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-08-25T20:12:02.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary2': 2024-08-25T20:12:02.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:12:02.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:12:02.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:12:02.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1fc75d837638 2024-08-25T20:12:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1fc75d837638 2024-08-25T20:12:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:12:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:12:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:12:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:12:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:12:01 2024 2024-08-25T20:12:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:12:01 2024 2024-08-25T20:12:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:12:01 2024 2024-08-25T20:12:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:12:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:12:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 89d01a96-1a53-44bb-b8a5-c17d22e98de0 2024-08-25T20:12:02.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:12:02.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:12:02.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:12:02.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-08-25T20:12:02.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd1 2024-08-25T20:12:02.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:12:02.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-08-25T20:12:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:12:02.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:12:02.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:12:02.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: e9345e44-7a64-42ad-8254-bc383712531d 2024-08-25T20:12:02.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:12:02.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:12:02.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:02.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:12:02.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:12:02.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:12:02.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:12:02.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:02.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary2 2024-08-25T20:12:02.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-08-25T20:12:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:12:02.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary2 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:02.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:12:02.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:12:02.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary3 2024-08-25T20:12:02.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-08-25T20:12:02.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:02.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:02.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:02.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:02.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:02.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:02.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:02.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:12:02.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:02.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-08-25T20:12:02.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary2 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary2 2024-08-25T20:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-08-25T20:12:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:12:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:12:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:12:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-08-25T20:12:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-08-25T20:12:02.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:12:02.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-08-25T20:12:03.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 996 2024-08-25T20:12:03.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:12:03.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:03.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-08-25T20:12:03.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary3': 2024-08-25T20:12:03.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:12:03.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:12:03.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:12:03.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1fd6e874f212 2024-08-25T20:12:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1fd6e874f212 2024-08-25T20:12:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:12:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:12:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:12:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:12:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:12:02 2024 2024-08-25T20:12:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:12:02 2024 2024-08-25T20:12:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:12:02 2024 2024-08-25T20:12:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:12:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:12:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 31496f9b-5900-44ec-b871-36538a41e65c 2024-08-25T20:12:03.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:12:03.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:12:03.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:12:03.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-08-25T20:12:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd2 2024-08-25T20:12:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:12:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-08-25T20:12:03.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:12:03.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:12:03.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:12:03.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: ef3e3879-5712-4ebe-9183-f919d40b8866 2024-08-25T20:12:03.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:12:03.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:12:03.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:03.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:12:03.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:12:03.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:12:03.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:12:03.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:03.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary3 2024-08-25T20:12:03.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-08-25T20:12:03.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:12:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary3 2024-08-25T20:12:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:12:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-08-25T20:12:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary3 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary4 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary3 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-08-25T20:12:03.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:12:03.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:03.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:12:03.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:12:03.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-08-25T20:12:03.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:03.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:03.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:03.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:03.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-08-25T20:12:04.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:12:04.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-08-25T20:12:04.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:12:04.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 998 2024-08-25T20:12:04.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-08-25T20:12:04.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:04.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary4': 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1fe276ec3884 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1fe276ec3884 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:12:03 2024 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:12:03 2024 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:12:03 2024 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:12:04.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:12:04.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 4f7fa68b-ea47-408d-8470-47214ea59e67 2024-08-25T20:12:04.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:12:04.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:12:04.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:12:04.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-08-25T20:12:04.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd3 2024-08-25T20:12:04.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:12:04.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-08-25T20:12:04.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:12:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:12:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:12:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: d4184536-0dcc-4d71-9065-f471808f61dc 2024-08-25T20:12:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:12:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:12:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:12:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:12:04.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:12:04.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:12:04.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary4 2024-08-25T20:12:04.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-08-25T20:12:04.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:12:04.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary4 2024-08-25T20:12:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:12:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-08-25T20:12:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:12:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary4 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary5 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary4 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:12:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:12:04.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:04.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-08-25T20:12:04.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:12:04.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:04.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:12:04.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:12:04.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-08-25T20:12:04.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:04.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:04.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:04.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:04.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-08-25T20:12:05.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:12:05.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-08-25T20:12:05.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:12:05.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1000 2024-08-25T20:12:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:05.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-08-25T20:12:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary5': 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 1ff31b2bb03b 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.1ff31b2bb03b 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:12:04 2024 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:12:04 2024 2024-08-25T20:12:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:12:04 2024 2024-08-25T20:12:05.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:12:05.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:12:05.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 54e7d5c4-e127-461b-8dd6-28ec2ebe7b7a 2024-08-25T20:12:05.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:12:05.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:12:05.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:12:05.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-08-25T20:12:05.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd4 2024-08-25T20:12:05.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:12:05.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-08-25T20:12:05.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:12:05.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:12:05.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:12:05.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 25680880-10da-484a-9115-8feeb098c1f7 2024-08-25T20:12:05.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:12:05.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:12:05.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:05.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:12:05.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:12:05.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:12:05.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:12:05.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:05.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary5 2024-08-25T20:12:05.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-08-25T20:12:05.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:12:05.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary5 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:12:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary6 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary5 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-08-25T20:12:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary5 2024-08-25T20:12:05.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:05.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:05.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-08-25T20:12:05.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:05.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:05.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:05.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:05.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:12:05.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:12:05.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:12:05.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:05.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-08-25T20:12:05.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:12:05.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:05.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:12:05.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:12:05.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-08-25T20:12:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-08-25T20:12:05.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:12:05.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-08-25T20:12:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:12:06.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1003 2024-08-25T20:12:06.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1002 2024-08-25T20:12:06.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:06.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:06.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-08-25T20:12:06.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:06.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:06.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:06.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:06.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:06.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary6': 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2005c366e79d 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2005c366e79d 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:12:05 2024 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:12:05 2024 2024-08-25T20:12:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:12:05 2024 2024-08-25T20:12:06.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:12:06.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:12:06.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 0a0d8e9d-63de-44ab-bdb2-17b97792d7d0 2024-08-25T20:12:06.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:12:06.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:12:06.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:12:06.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-08-25T20:12:06.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd5 2024-08-25T20:12:06.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:12:06.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-08-25T20:12:06.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:12:06.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:12:06.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:12:06.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 82dad50e-b2e8-4dc3-ac21-3a68808f8dc8 2024-08-25T20:12:06.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:12:06.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:12:06.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:06.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:12:06.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:12:06.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:12:06.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:12:06.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:06.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary6 2024-08-25T20:12:06.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-08-25T20:12:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:12:06.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary6 2024-08-25T20:12:06.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:12:06.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-08-25T20:12:06.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:06.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary7 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary6 2024-08-25T20:12:06.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary6 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:06.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:06.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:06.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:12:06.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:12:06.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:12:06.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:06.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-08-25T20:12:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:12:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:12:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:12:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-08-25T20:12:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-08-25T20:12:06.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:12:06.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-08-25T20:12:07.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:12:07.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1005 2024-08-25T20:12:07.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1006 2024-08-25T20:12:07.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:07.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-08-25T20:12:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:07.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary7': 2024-08-25T20:12:07.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2017a81b6089 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2017a81b6089 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:12:06 2024 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:12:06 2024 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:12:06 2024 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:12:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: ce7df407-6dd2-4d59-8f98-8c52eaefd177 2024-08-25T20:12:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:12:07.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:12:07.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:12:07.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-08-25T20:12:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd6 2024-08-25T20:12:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:12:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-08-25T20:12:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:12:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:12:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:12:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 097d6951-2769-48f9-829c-59357369dbd0 2024-08-25T20:12:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:12:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:12:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:12:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:12:07.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:12:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:12:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary7 2024-08-25T20:12:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-08-25T20:12:07.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:12:07.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary7 2024-08-25T20:12:07.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:12:07.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-08-25T20:12:07.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:07.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:07.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:07.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary8 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary7 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary7 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:07.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:07.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:12:07.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:12:07.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:12:07.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:07.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-08-25T20:12:07.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:12:07.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:07.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:12:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:12:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-08-25T20:12:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-08-25T20:12:07.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:12:07.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-08-25T20:12:08.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:12:08.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1009 2024-08-25T20:12:08.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1008 2024-08-25T20:12:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-08-25T20:12:08.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:12:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:12:08.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary8': 2024-08-25T20:12:08.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2029949eefd2 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2029949eefd2 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:12:07 2024 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:12:07 2024 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:12:07 2024 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:12:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: cf4f7205-657c-4d10-848c-7bf5c56279da 2024-08-25T20:12:08.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:12:08.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:12:08.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:12:08.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-08-25T20:12:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd7 2024-08-25T20:12:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:12:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-08-25T20:12:08.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:12:08.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:12:08.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:12:08.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: e1cf471d-e4b4-41ef-87dd-911e95ebde54 2024-08-25T20:12:08.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:12:08.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:12:08.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:08.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:12:08.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:12:08.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:12:08.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:12:08.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:08.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary8 2024-08-25T20:12:08.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-08-25T20:12:08.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:12:08.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary8 2024-08-25T20:12:08.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:12:08.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-08-25T20:12:08.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:08.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary9 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary8 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary8 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-08-25T20:12:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:12:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:12:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:12:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-08-25T20:12:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:12:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:12:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:12:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-08-25T20:12:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:08.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:08.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-08-25T20:12:09.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:12:09.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-08-25T20:12:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1011 2024-08-25T20:12:09.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:12:09.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1012 2024-08-25T20:12:09.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:09.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:09.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:12:09.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:09.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:09.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:09.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:12:09.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:09.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:09.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:09.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:09.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:09.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:09.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:09.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-08-25T20:12:09.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary9': 2024-08-25T20:12:09.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:12:09.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:12:09.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:12:09.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 203b87aa161a 2024-08-25T20:12:09.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.203b87aa161a 2024-08-25T20:12:09.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:12:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:12:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:12:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:12:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:12:08 2024 2024-08-25T20:12:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:12:08 2024 2024-08-25T20:12:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:12:08 2024 2024-08-25T20:12:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:12:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:12:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 975cc06f-b335-4181-8f15-5fd979f23a46 2024-08-25T20:12:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:12:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:12:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:12:09.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-08-25T20:12:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd8 2024-08-25T20:12:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:12:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-08-25T20:12:09.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:12:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:12:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:12:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: debd3340-7f7d-44e2-9486-f55105435793 2024-08-25T20:12:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:12:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:12:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:12:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:12:09.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:12:09.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:12:09.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:09.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary9 2024-08-25T20:12:09.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-08-25T20:12:09.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:12:09.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary9 2024-08-25T20:12:09.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:12:09.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-08-25T20:12:09.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:09.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:09.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary10 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary9 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:09.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary9 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:09.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:09.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:09.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:12:09.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:12:09.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:12:09.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:12:09.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-08-25T20:12:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:12:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:12:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:12:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-08-25T20:12:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:12:09.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-08-25T20:12:09.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:12:09.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-08-25T20:12:10.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:10.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:10.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:10.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:10.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:10.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:10.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:12:10.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:10.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:10.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:10.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:12:10.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:12:10.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1014 2024-08-25T20:12:10.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1015 2024-08-25T20:12:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1016 2024-08-25T20:12:10.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-08-25T20:12:10.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:10.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:10.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:10.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary10': 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 204f6e273479 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.204f6e273479 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:12:09 2024 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:12:09 2024 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:12:09 2024 2024-08-25T20:12:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:12:10.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:12:10.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: dda4aac1-b2d6-46d3-a987-574494886b5c 2024-08-25T20:12:10.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:12:10.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:12:10.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:12:10.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-08-25T20:12:10.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd9 2024-08-25T20:12:10.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:12:10.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-08-25T20:12:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:12:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:12:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:12:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: c85e4563-3c11-4351-9081-641dbefb178b 2024-08-25T20:12:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:12:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:12:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:12:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:12:10.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:12:10.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:12:10.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:12:10.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary10 2024-08-25T20:12:10.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-08-25T20:12:10.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:12:10.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary10 2024-08-25T20:12:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:12:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-08-25T20:12:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:12:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:12:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:12:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary10 2024-08-25T20:12:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:12:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59075 2024-08-25T20:12:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary10 2024-08-25T20:12:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-08-25T20:12:11.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:12:11.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-08-25T20:12:11.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:11.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:12:11.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:11.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:11.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:11.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:12:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:11.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:11.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:11.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:11.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:11.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:11.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1019 2024-08-25T20:12:11.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1020 2024-08-25T20:12:11.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:11.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1018 2024-08-25T20:12:11.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:11.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:12.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:12.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:12:12.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:12.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:12.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:12.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:12:12.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:12.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:12:12.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:12.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:12.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:12.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:12:12.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1022 2024-08-25T20:12:12.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1021 2024-08-25T20:12:12.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:12.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:12:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:13.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:13.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:12:13.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:13.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:13.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:13.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:13.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:13.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:13.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:13.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:12:13.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:13.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:13.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:13.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:12:13.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1024 2024-08-25T20:12:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1023 2024-08-25T20:12:13.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:13.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:14.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1026 2024-08-25T20:12:14.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1025 2024-08-25T20:12:14.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1027 2024-08-25T20:12:14.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:14.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:14.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:14.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:12:14.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:14.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:14.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:14.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:12:14.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:14.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:12:14.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:14.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:14.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:14.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:12:14.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:14.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:14.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:14.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:14.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:14.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:15.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1030 2024-08-25T20:12:15.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1028 2024-08-25T20:12:15.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1029 2024-08-25T20:12:15.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:15.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:15.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:15.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:15.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:12:15.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:15.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:15.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:15.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:12:15.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:15.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:15.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:15.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:15.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:15.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:12:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:12:16.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:16.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:12:16.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:16.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:16.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:16.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:12:16.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1031 2024-08-25T20:12:16.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1032 2024-08-25T20:12:16.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:16.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:17.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:17.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:12:17.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:17.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:17.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:17.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:12:17.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:17.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:12:17.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:17.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:17.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:17.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:12:17.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:17.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:17.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:17.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:17.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:17.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:17.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1033 2024-08-25T20:12:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1034 2024-08-25T20:12:17.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:17.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:18.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:18.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:18.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:18.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:18.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:18.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:18.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:18.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:12:18.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:18.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:18.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:18.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:12:18.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:18.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:12:18.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:18.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:18.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:18.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:12:18.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1036 2024-08-25T20:12:18.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1035 2024-08-25T20:12:18.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:18.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:18.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1037 2024-08-25T20:12:18.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:19.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:19.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:12:19.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:19.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:19.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:19.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:12:19.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:19.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1039 2024-08-25T20:12:19.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1040 2024-08-25T20:12:19.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1038 2024-08-25T20:12:19.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:19.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:20.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1041 2024-08-25T20:12:20.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:20.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:12:20.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:20.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:20.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:20.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:12:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:12:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:12:20.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1042 2024-08-25T20:12:20.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:20.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:21.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1043 2024-08-25T20:12:21.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:21.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1044 2024-08-25T20:12:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:21.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:21.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:12:21.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:21.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:21.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:21.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:12:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:12:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:12:21.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:21.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:21.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:21.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:21.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:21.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1045 2024-08-25T20:12:22.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:22.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1047 2024-08-25T20:12:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:12:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:12:22.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:12:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:22.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:22.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:12:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1048 2024-08-25T20:12:23.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1050 2024-08-25T20:12:23.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:23.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:23.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:23.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:12:23.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:23.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:23.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:23.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:12:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:12:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:12:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1046 2024-08-25T20:12:24.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:24.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:12:24.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:24.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:24.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:24.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:12:24.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:24.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1051 2024-08-25T20:12:24.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:25.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1049 2024-08-25T20:12:25.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:25.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1053 2024-08-25T20:12:25.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:12:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:12:25.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:12:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:12:26.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1052 2024-08-25T20:12:26.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:26.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1055 2024-08-25T20:12:26.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:12:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:12:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:12:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:12:27.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1054 2024-08-25T20:12:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:27.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1057 2024-08-25T20:12:27.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:27.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:27.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:27.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:27.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:27.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:27.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:27.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:27.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:12:27.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:27.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:27.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:27.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:12:28.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1056 2024-08-25T20:12:28.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:28.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1059 2024-08-25T20:12:28.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:28.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:28.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:28.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:28.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:28.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:28.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:28.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:28.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:12:28.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:28.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:28.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:28.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:12:29.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1058 2024-08-25T20:12:29.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:29.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:29.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:29.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:29.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:29.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:29.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:29.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:29.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:12:29.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:29.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:29.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:29.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:12:29.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1061 2024-08-25T20:12:29.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:30.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1060 2024-08-25T20:12:30.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:30.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:12:30.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:30.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:30.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:30.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:12:30.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:30.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:30.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:12:30.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:30.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:30.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:30.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:12:30.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1063 2024-08-25T20:12:30.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:31.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1062 2024-08-25T20:12:31.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:31.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:12:31.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:31.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:31.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:31.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:12:31.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1065 2024-08-25T20:12:31.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:32.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1064 2024-08-25T20:12:32.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:32.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:12:32.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:32.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:32.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:32.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:12:32.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:32.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1067 2024-08-25T20:12:32.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:32.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:32.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:32.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:32.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:32.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:32.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:33.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1066 2024-08-25T20:12:33.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:33.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:12:33.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:33.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:33.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:33.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:12:33.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:33.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1069 2024-08-25T20:12:33.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:33.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:33.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:34.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1068 2024-08-25T20:12:34.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:12:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:12:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:34.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1071 2024-08-25T20:12:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:12:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:12:35.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1070 2024-08-25T20:12:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:12:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:12:35.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:35.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:35.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:12:35.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:35.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:35.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:35.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:12:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1073 2024-08-25T20:12:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:36.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1072 2024-08-25T20:12:36.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:36.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:12:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:12:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:12:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:36.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:36.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:12:36.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1075 2024-08-25T20:12:36.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:37.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1074 2024-08-25T20:12:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:37.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:12:37.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:37.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:37.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:37.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:12:37.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1077 2024-08-25T20:12:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:38.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1076 2024-08-25T20:12:38.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:38.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:38.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:38.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:38.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:38.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:38.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:38.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:38.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:12:38.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:38.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:38.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:38.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:12:38.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1079 2024-08-25T20:12:38.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:39.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1078 2024-08-25T20:12:39.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:39.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:39.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:12:39.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:39.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:39.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:39.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:12:39.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1081 2024-08-25T20:12:39.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1080 2024-08-25T20:12:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:12:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:12:40.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:40.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1083 2024-08-25T20:12:40.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:40.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:12:40.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:40.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:40.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:40.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:12:40.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:41.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1082 2024-08-25T20:12:41.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:12:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:12:41.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1085 2024-08-25T20:12:41.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:41.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:41.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:41.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:41.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:41.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:41.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:42.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1084 2024-08-25T20:12:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:12:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:12:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1087 2024-08-25T20:12:42.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1086 2024-08-25T20:12:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:12:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:12:43.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:43.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1089 2024-08-25T20:12:43.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:43.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:43.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:43.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:43.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:43.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:44.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:44.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:12:44.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:44.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:44.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:44.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:12:44.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1088 2024-08-25T20:12:44.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:44.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:44.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:12:44.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:44.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:44.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:44.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:12:44.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1091 2024-08-25T20:12:44.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:45.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1090 2024-08-25T20:12:45.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:45.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:12:45.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:45.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:45.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:45.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:12:45.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:45.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1093 2024-08-25T20:12:45.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:45.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:45.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:12:45.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:45.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:45.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:45.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:12:46.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1092 2024-08-25T20:12:46.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1095 2024-08-25T20:12:46.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:46.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:46.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:12:46.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:46.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:46.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:46.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:12:46.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:46.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:12:46.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:46.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:46.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:46.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:12:47.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1094 2024-08-25T20:12:47.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:47.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:47.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:47.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:47.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:47.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:47.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:12:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:12:47.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1097 2024-08-25T20:12:47.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:48.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1096 2024-08-25T20:12:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:48.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:48.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:48.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:12:48.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:48.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:48.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:48.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:12:48.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1099 2024-08-25T20:12:48.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:49.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:49.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1098 2024-08-25T20:12:49.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:49.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:12:49.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:49.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:49.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:49.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:12:49.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:49.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1101 2024-08-25T20:12:49.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1100 2024-08-25T20:12:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:12:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:12:50.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:50.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1103 2024-08-25T20:12:50.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:50.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:50.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:12:50.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:50.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:50.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:50.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:12:51.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1102 2024-08-25T20:12:51.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:51.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1105 2024-08-25T20:12:51.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:51.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:51.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:12:51.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:51.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:51.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:51.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:12:51.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:51.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:51.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:51.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:51.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:51.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:52.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1104 2024-08-25T20:12:52.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:52.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1107 2024-08-25T20:12:52.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:12:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:52.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:12:52.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:52.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:52.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:52.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:52.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:52.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:53.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1106 2024-08-25T20:12:53.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:53.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1109 2024-08-25T20:12:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:12:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:12:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:53.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:53.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:53.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:53.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:53.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:54.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1108 2024-08-25T20:12:54.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:12:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:12:54.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:54.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:12:54.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:54.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:54.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:12:54.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:12:54.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1111 2024-08-25T20:12:54.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:55.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1110 2024-08-25T20:12:55.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:55.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:12:55.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:55.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:55.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:55.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:12:55.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:55.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1113 2024-08-25T20:12:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:12:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:12:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:12:55.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:56.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1112 2024-08-25T20:12:56.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:56.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:12:56.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:56.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:56.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:12:56.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:12:56.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:12:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:12:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:12:56.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1115 2024-08-25T20:12:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:57.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1114 2024-08-25T20:12:57.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:57.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:12:57.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:57.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:57.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:12:57.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:12:57.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:57.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1117 2024-08-25T20:12:57.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:12:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:12:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:12:58.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1116 2024-08-25T20:12:58.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:12:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:12:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:12:58.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1119 2024-08-25T20:12:58.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:58.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:58.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:12:58.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:58.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:58.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:12:58.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:12:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1118 2024-08-25T20:12:59.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:59.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:59.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:12:59.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:59.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:59.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:12:59.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:12:59.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1121 2024-08-25T20:12:59.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:12:59.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:12:59.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:12:59.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:12:59.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:12:59.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:12:59.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:00.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1120 2024-08-25T20:13:00.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:00.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:13:00.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:00.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:00.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:13:00.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:13:00.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1123 2024-08-25T20:13:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:13:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:13:00.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:13:00.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:01.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1122 2024-08-25T20:13:01.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:01.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1125 2024-08-25T20:13:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:13:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:13:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:13:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:13:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:13:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:13:02.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1124 2024-08-25T20:13:02.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:02.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:02.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:13:02.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:02.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:02.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:13:02.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:13:02.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1127 2024-08-25T20:13:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:13:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:13:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:13:02.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:03.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1126 2024-08-25T20:13:03.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:03.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:13:03.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:03.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:03.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:13:03.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:13:03.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:03.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1129 2024-08-25T20:13:03.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:03.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:03.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:13:03.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:03.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:03.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:13:03.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:13:04.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:04.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:13:04.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:04.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:04.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:13:04.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:13:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1128 2024-08-25T20:13:04.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:04.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:04.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:13:04.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:04.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:04.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:13:04.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:13:04.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1131 2024-08-25T20:13:04.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:05.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1130 2024-08-25T20:13:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:13:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:13:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:05.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1133 2024-08-25T20:13:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:13:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:05.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:13:05.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:13:05.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:06.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1132 2024-08-25T20:13:06.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1135 2024-08-25T20:13:06.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:06.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:13:06.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:06.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:06.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:13:06.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:13:06.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:06.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:06.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:13:06.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:06.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:06.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:13:06.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:13:07.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1134 2024-08-25T20:13:07.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:07.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:13:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:13:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:13:07.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:07.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1137 2024-08-25T20:13:07.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:07.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:13:07.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:13:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:13:07.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:08.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1136 2024-08-25T20:13:08.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:08.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:13:08.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:08.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:08.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:13:08.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:13:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:13:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:13:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:13:08.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1139 2024-08-25T20:13:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:13:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:13:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:13:09.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1138 2024-08-25T20:13:09.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:09.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:13:09.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:09.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:09.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:13:09.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:09.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1141 2024-08-25T20:13:09.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:10.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1140 2024-08-25T20:13:10.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:13:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:13:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:13:10.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1143 2024-08-25T20:13:10.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:10.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:13:10.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:10.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:10.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:13:10.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:13:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:11.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1142 2024-08-25T20:13:11.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:11.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1145 2024-08-25T20:13:11.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:13:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:13:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:13:11.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:11.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:13:11.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:11.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:11.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:13:11.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:13:12.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1144 2024-08-25T20:13:12.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:12.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:12.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:13:12.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:12.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:12.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:13:12.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:13:12.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1147 2024-08-25T20:13:12.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:12.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:13:12.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:12.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:12.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:13:12.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:13:12.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:13.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1146 2024-08-25T20:13:13.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:13.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:13:13.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:13.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:13.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:13:13.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:13:13.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:13.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1149 2024-08-25T20:13:13.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:13:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:13:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:13:13.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:14.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:14.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:13:14.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:14.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:14.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:13:14.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:13:14.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1148 2024-08-25T20:13:14.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:14.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:13:14.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:14.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:14.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:13:14.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:13:14.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:14.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1151 2024-08-25T20:13:14.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:15.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1150 2024-08-25T20:13:15.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:15.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:13:15.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:15.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:15.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:13:15.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:15.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:15.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1153 2024-08-25T20:13:15.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:13:15.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:15.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:15.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:13:15.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:13:16.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1152 2024-08-25T20:13:16.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1155 2024-08-25T20:13:16.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:16.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:16.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:13:16.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:16.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:16.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:13:16.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:13:16.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:16.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:13:16.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:16.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:16.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:13:16.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:13:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1154 2024-08-25T20:13:17.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:17.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:13:17.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:17.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:17.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:13:17.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:13:17.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:17.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:17.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:13:17.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:17.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:17.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:13:17.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:13:17.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1157 2024-08-25T20:13:17.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:18.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1156 2024-08-25T20:13:18.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:18.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:13:18.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:18.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:18.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:13:18.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:13:18.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:18.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:18.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:13:18.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:18.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:18.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:13:18.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:13:18.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1159 2024-08-25T20:13:18.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:19.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:19.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:13:19.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:19.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:19.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:13:19.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:13:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:13:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:13:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:19.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1158 2024-08-25T20:13:19.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:19.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1161 2024-08-25T20:13:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:20.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1160 2024-08-25T20:13:20.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:20.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:13:20.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:20.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:20.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:13:20.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:13:20.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:20.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1163 2024-08-25T20:13:20.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:13:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:13:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:13:21.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1162 2024-08-25T20:13:21.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1165 2024-08-25T20:13:21.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:21.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:21.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:13:21.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:21.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:21.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:13:21.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:13:21.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:21.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:13:21.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:21.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:21.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:13:21.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:13:22.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1164 2024-08-25T20:13:22.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:22.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1167 2024-08-25T20:13:22.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:22.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:22.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:13:22.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:22.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:22.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:13:22.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:13:22.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:22.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:13:22.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:22.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:22.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:13:22.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:13:23.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1166 2024-08-25T20:13:23.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:23.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1169 2024-08-25T20:13:23.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:23.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:23.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:13:23.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:23.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:23.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:13:23.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:13:23.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:23.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:13:23.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:23.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:23.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:13:23.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:13:24.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1168 2024-08-25T20:13:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:13:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:24.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:13:24.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:13:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:24.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1171 2024-08-25T20:13:24.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:24.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:24.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:13:24.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:24.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:24.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:13:24.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:13:25.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1170 2024-08-25T20:13:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:13:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:13:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:25.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:25.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1173 2024-08-25T20:13:25.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:25.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:13:25.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:25.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:25.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:13:25.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:13:25.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:26.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1172 2024-08-25T20:13:26.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:26.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:26.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:13:26.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:26.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:26.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:13:26.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:13:26.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1175 2024-08-25T20:13:26.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:26.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:26.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:13:26.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:26.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:26.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:13:26.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:13:27.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:27.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:13:27.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:27.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:27.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:13:27.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:13:27.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1174 2024-08-25T20:13:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:13:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:27.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:13:27.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:13:27.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:27.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1177 2024-08-25T20:13:27.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:28.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1176 2024-08-25T20:13:28.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:28.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:13:28.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:28.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:13:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:13:28.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:28.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:28.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:13:28.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:28.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:28.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:13:28.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:13:28.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1179 2024-08-25T20:13:28.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:29.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1178 2024-08-25T20:13:29.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:29.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:13:29.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:29.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:29.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:13:29.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:13:29.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:29.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:13:29.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:29.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:29.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:13:29.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:29.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:29.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1181 2024-08-25T20:13:29.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:30.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1180 2024-08-25T20:13:30.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:13:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:13:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:13:30.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1183 2024-08-25T20:13:30.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:30.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:30.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:13:30.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:30.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:30.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:13:30.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:13:31.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1182 2024-08-25T20:13:31.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:31.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1185 2024-08-25T20:13:31.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:31.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:13:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:13:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:13:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:13:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:13:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:13:32.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1184 2024-08-25T20:13:32.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:32.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:13:32.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:32.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:32.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:13:32.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:13:32.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:32.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1187 2024-08-25T20:13:32.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:32.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:13:32.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:32.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:32.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:13:32.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:13:32.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:33.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1186 2024-08-25T20:13:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:13:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:13:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:13:33.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:33.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1189 2024-08-25T20:13:33.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:33.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:13:33.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:33.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:33.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:13:33.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:13:33.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:34.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:13:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:13:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:13:34.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1188 2024-08-25T20:13:34.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:34.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:13:34.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:34.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:34.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:13:34.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:13:34.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:34.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1191 2024-08-25T20:13:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:35.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1190 2024-08-25T20:13:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:35.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:35.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:13:35.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:35.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:35.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:13:35.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:35.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1193 2024-08-25T20:13:35.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:35.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:35.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:13:35.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:35.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:35.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:13:35.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:13:36.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1192 2024-08-25T20:13:36.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:36.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1195 2024-08-25T20:13:36.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:36.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:36.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:13:36.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:36.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:36.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:13:36.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:13:36.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:36.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:13:36.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:36.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:36.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:13:36.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:13:37.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1194 2024-08-25T20:13:37.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:37.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1197 2024-08-25T20:13:37.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:37.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:37.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:13:37.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:37.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:37.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:13:37.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:13:37.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:37.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:13:37.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:37.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:37.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:13:37.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:13:38.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1196 2024-08-25T20:13:38.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:38.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1199 2024-08-25T20:13:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:38.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:38.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:13:38.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:38.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:38.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:13:38.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:13:38.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:38.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:13:38.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:38.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:38.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:13:38.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:13:39.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1198 2024-08-25T20:13:39.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:39.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:39.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:13:39.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:39.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:39.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:13:39.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:13:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1201 2024-08-25T20:13:39.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:39.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:39.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:13:39.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:39.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:39.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:13:39.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:40.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1200 2024-08-25T20:13:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:13:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:13:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:13:40.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:40.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1203 2024-08-25T20:13:40.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:40.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:13:40.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:40.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:40.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:13:40.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:13:40.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:41.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1202 2024-08-25T20:13:41.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:41.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:13:41.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:41.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:41.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:13:41.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:13:41.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:41.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:41.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:13:41.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:41.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:41.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:13:41.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:13:41.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1205 2024-08-25T20:13:41.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1204 2024-08-25T20:13:42.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:42.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:13:42.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:42.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:42.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:13:42.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:13:42.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:42.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1207 2024-08-25T20:13:42.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:42.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:13:42.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:42.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:42.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:13:42.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:13:42.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:43.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1206 2024-08-25T20:13:43.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:43.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:13:43.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:43.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:43.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:13:43.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:13:43.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:43.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1209 2024-08-25T20:13:43.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:43.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:13:43.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:43.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:43.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:13:43.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:13:43.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1208 2024-08-25T20:13:44.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:13:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:44.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:44.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:13:44.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:13:44.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1211 2024-08-25T20:13:44.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:44.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:44.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:13:44.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:44.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:44.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:13:44.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:13:45.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1210 2024-08-25T20:13:45.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:13:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:13:45.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:45.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:45.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1213 2024-08-25T20:13:45.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:45.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:13:45.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:45.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:45.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:13:45.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:13:45.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:46.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1212 2024-08-25T20:13:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:46.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:46.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:13:46.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:13:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:13:46.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1215 2024-08-25T20:13:46.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:13:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:13:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:13:47.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1214 2024-08-25T20:13:47.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:47.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:13:47.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:47.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:47.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:13:47.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:13:47.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:13:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:13:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:13:47.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1217 2024-08-25T20:13:47.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:48.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1216 2024-08-25T20:13:48.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:48.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:13:48.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:48.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:48.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:13:48.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:13:48.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:48.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:48.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:13:48.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:48.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:48.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:13:48.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:13:48.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1219 2024-08-25T20:13:48.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:49.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:49.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:13:49.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:49.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:49.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:13:49.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:13:49.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1218 2024-08-25T20:13:49.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:49.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:13:49.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:49.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:49.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:13:49.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:49.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1221 2024-08-25T20:13:49.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:50.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1220 2024-08-25T20:13:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:13:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:13:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:13:50.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:50.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1223 2024-08-25T20:13:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:13:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:13:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:13:50.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:51.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1222 2024-08-25T20:13:51.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:51.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1225 2024-08-25T20:13:51.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:51.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:13:51.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:51.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:51.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:13:51.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:13:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:51.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:51.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:13:51.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:51.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:51.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:13:51.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:13:52.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1224 2024-08-25T20:13:52.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:52.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:13:52.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:52.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:52.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:13:52.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:13:52.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:52.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1227 2024-08-25T20:13:52.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:52.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:13:52.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:52.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:52.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:13:52.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:13:52.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:53.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1226 2024-08-25T20:13:53.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:53.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:13:53.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:53.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:53.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:13:53.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:13:53.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:53.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1229 2024-08-25T20:13:53.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:53.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:13:53.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:53.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:53.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:13:53.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:13:53.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:13:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:13:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:13:54.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:54.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:13:54.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:54.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:54.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:13:54.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:13:54.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1228 2024-08-25T20:13:54.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:54.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1231 2024-08-25T20:13:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:55.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1230 2024-08-25T20:13:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:13:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:13:55.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:55.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:55.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:55.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:13:55.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:55.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:55.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:13:55.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:13:55.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1233 2024-08-25T20:13:55.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:56.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1232 2024-08-25T20:13:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:56.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1235 2024-08-25T20:13:56.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:13:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:13:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:13:56.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:56.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:13:56.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:56.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:56.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:13:56.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:13:57.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1234 2024-08-25T20:13:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:57.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:57.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:13:57.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:57.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:57.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:13:57.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:13:57.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1237 2024-08-25T20:13:57.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:57.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:57.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:13:57.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:57.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:57.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:13:57.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:13:58.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1236 2024-08-25T20:13:58.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:58.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:13:58.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:58.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:58.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:13:58.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:13:58.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:58.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1239 2024-08-25T20:13:58.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:58.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:13:58.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:58.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:58.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:13:58.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:13:58.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:59.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:59.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:13:59.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:59.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:59.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:13:59.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:13:59.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:13:59.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:13:59.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:13:59.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:13:59.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:13:59.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:13:59.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1238 2024-08-25T20:13:59.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:13:59.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1241 2024-08-25T20:13:59.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:00.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1240 2024-08-25T20:14:00.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:00.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:14:00.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:00.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:00.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:14:00.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:14:00.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:14:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:14:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:14:00.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1243 2024-08-25T20:14:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:01.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1242 2024-08-25T20:14:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:01.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1245 2024-08-25T20:14:01.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:01.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:01.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:14:01.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:01.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:14:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:14:01.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:01.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:14:01.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:01.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:01.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:14:01.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:14:02.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1244 2024-08-25T20:14:02.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1247 2024-08-25T20:14:02.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:14:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:02.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:02.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:14:02.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:14:02.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:02.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:14:02.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:02.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:02.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:14:02.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:14:03.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1246 2024-08-25T20:14:03.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:03.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1249 2024-08-25T20:14:03.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:03.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:03.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:14:03.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:03.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:03.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:14:03.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:14:03.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:03.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:14:03.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:03.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:03.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:14:03.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:14:04.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1248 2024-08-25T20:14:04.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:04.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:14:04.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:04.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:04.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:14:04.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:14:04.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:04.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1251 2024-08-25T20:14:04.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:04.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:14:04.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:04.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:04.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:14:04.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:14:04.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:05.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1250 2024-08-25T20:14:05.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:05.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:14:05.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:05.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:05.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:14:05.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:14:05.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:05.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:05.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:14:05.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1253 2024-08-25T20:14:05.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:05.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:05.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:14:05.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:14:05.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:06.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1252 2024-08-25T20:14:06.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:14:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:14:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:14:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:14:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:14:06.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:14:06.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1255 2024-08-25T20:14:06.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:07.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1254 2024-08-25T20:14:07.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:07.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:14:07.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:07.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:07.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:14:07.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:14:07.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:07.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:07.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:14:07.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:07.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:07.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:14:07.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:14:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1257 2024-08-25T20:14:07.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:08.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1256 2024-08-25T20:14:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:14:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:14:08.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:14:08.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:08.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1259 2024-08-25T20:14:08.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:08.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:14:08.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:08.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:08.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:14:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:14:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:09.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:09.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:14:09.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:09.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:09.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:14:09.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:14:09.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1258 2024-08-25T20:14:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1261 2024-08-25T20:14:09.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:09.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:09.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:14:09.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:09.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:09.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:14:09.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:14:10.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1260 2024-08-25T20:14:10.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:10.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:14:10.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:10.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:10.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:14:10.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:14:10.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:10.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1263 2024-08-25T20:14:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:14:10.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:10.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:10.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:14:10.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:14:10.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:11.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1262 2024-08-25T20:14:11.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:11.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1265 2024-08-25T20:14:11.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:11.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:11.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:14:11.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:11.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:11.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:14:11.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:14:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:14:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:14:11.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:14:12.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1264 2024-08-25T20:14:12.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:12.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:14:12.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:12.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:12.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:14:12.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:14:12.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:12.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1267 2024-08-25T20:14:12.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:12.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:14:12.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:12.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:12.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:14:12.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:14:12.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:13.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1266 2024-08-25T20:14:13.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:13.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:14:13.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:13.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:13.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:14:13.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:14:13.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:13.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1269 2024-08-25T20:14:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:14:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:13.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:13.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:14:13.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:14:13.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:14:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:14:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:14:14.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1268 2024-08-25T20:14:14.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:14.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:14:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:14:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:14:14.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:14.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1271 2024-08-25T20:14:14.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:15.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1270 2024-08-25T20:14:15.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:15.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:14:15.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:15.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:15.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:14:15.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:14:15.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:14:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:14:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:14:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1273 2024-08-25T20:14:15.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:16.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1272 2024-08-25T20:14:16.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:16.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1275 2024-08-25T20:14:16.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:16.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:16.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:14:16.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:16.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:16.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:14:16.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:14:16.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:16.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:14:16.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:16.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:16.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:14:16.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:14:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1274 2024-08-25T20:14:17.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:17.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1277 2024-08-25T20:14:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:17.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:17.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:14:17.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:17.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:17.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:14:17.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:14:17.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:17.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:14:17.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:17.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:17.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:14:17.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:14:18.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1276 2024-08-25T20:14:18.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:18.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:14:18.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:18.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:18.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:14:18.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:14:18.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:18.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1279 2024-08-25T20:14:18.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:14:18.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:18.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:18.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:14:18.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:14:19.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:19.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:14:19.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:19.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:19.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:14:19.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:14:19.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:19.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:14:19.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:19.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:19.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:14:19.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:14:19.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1278 2024-08-25T20:14:19.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:19.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1281 2024-08-25T20:14:19.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:14:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:14:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:14:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:14:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:14:20.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:14:20.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1280 2024-08-25T20:14:20.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:20.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1283 2024-08-25T20:14:20.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:21.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1282 2024-08-25T20:14:21.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:21.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1285 2024-08-25T20:14:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:14:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:14:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:14:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:14:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:21.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:21.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:14:21.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:14:22.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1284 2024-08-25T20:14:22.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:22.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1287 2024-08-25T20:14:22.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:22.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:22.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:14:22.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:22.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:22.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:14:22.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:14:22.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:22.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:14:22.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:22.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:22.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:14:22.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:14:23.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1286 2024-08-25T20:14:23.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1289 2024-08-25T20:14:23.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:23.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:23.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:14:23.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:23.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:23.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:14:23.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:14:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:23.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:14:23.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:23.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:23.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:14:23.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:14:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1288 2024-08-25T20:14:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:14:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:14:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:14:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:24.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1291 2024-08-25T20:14:24.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:24.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:14:24.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:24.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:24.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:14:24.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:14:24.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:25.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1290 2024-08-25T20:14:25.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:25.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:14:25.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:25.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:25.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:14:25.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:14:25.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:25.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1293 2024-08-25T20:14:25.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:25.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:25.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:14:25.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:25.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:25.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:14:25.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:14:26.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1292 2024-08-25T20:14:26.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:26.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:26.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:14:26.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:26.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:26.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:14:26.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:14:26.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1295 2024-08-25T20:14:26.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:26.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:26.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:14:26.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:26.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:26.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:14:26.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:14:27.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1294 2024-08-25T20:14:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:14:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59207 2024-08-25T20:14:27.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:14:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:14:27.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:14:27.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1297 2024-08-25T20:14:27.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:28.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1296 2024-08-25T20:14:28.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:14:28.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59339 2024-08-25T20:14:28.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:28.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:28.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:14:28.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:28.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:28.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:14:28.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:14:29.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1298 2024-08-25T20:14:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:14:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59471 2024-08-25T20:14:29.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:29.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:14:29.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:29.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:29.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:14:29.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:14:29.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:30.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1299 2024-08-25T20:14:30.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:14:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59623 2024-08-25T20:14:30.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:14:30.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:14:30.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:14:30.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:14:30.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:14:30.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:14:31.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1300 2024-08-25T20:14:31.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:14:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59776 2024-08-25T20:14:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1301 2024-08-25T20:14:32.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:32.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:14:32.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59929 2024-08-25T20:14:33.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1302 2024-08-25T20:14:33.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:14:33.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 60082 2024-08-25T20:14:33.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:14:34.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:14:34.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 60255 2024-08-25T20:14:35.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:14:35.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 60429 2024-08-25T20:14:36.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:14:36.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:14:36.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59076 2024-08-25T20:17:01.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:17:01.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:17:01.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:17:01.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59208 2024-08-25T20:17:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:17:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:17:02.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:17:02.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59340 2024-08-25T20:17:04.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:17:04.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:17:04.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:17:04.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59472 2024-08-25T20:17:05.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:17:05.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:17:05.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:17:05.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59624 2024-08-25T20:17:06.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:17:06.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:17:06.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:17:06.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59777 2024-08-25T20:17:06.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:17:06.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:17:06.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:17:06.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 59930 2024-08-25T20:17:07.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:17:07.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:17:07.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:17:07.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 60083 2024-08-25T20:17:09.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:17:09.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:17:09.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:17:09.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 60256 2024-08-25T20:17:10.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:17:10.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:17:10.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:17:10.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 60430 2024-08-25T20:17:11.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:17:11.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:17:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:17:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:17:11.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:17:11.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 1 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd0 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary1 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary1 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary1 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 2 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd1 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary2 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary2 2024-08-25T20:17:11.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:17:11.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary2 2024-08-25T20:17:11.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:17:11.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:17:11.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 3 2024-08-25T20:17:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd2 2024-08-25T20:17:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary3 2024-08-25T20:17:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary3 2024-08-25T20:17:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:17:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary3 2024-08-25T20:17:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:17:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:17:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 4 2024-08-25T20:17:11.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd3 2024-08-25T20:17:11.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary4 2024-08-25T20:17:11.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary4 2024-08-25T20:17:11.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:17:11.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary4 2024-08-25T20:17:11.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:17:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:17:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 5 2024-08-25T20:17:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd4 2024-08-25T20:17:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary5 2024-08-25T20:17:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary5 2024-08-25T20:17:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:17:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary5 2024-08-25T20:17:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:17:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:17:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 6 2024-08-25T20:17:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd5 2024-08-25T20:17:11.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary6 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary6 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary6 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 7 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd6 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary7 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary7 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary7 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:17:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:17:11.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 8 2024-08-25T20:17:11.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd7 2024-08-25T20:17:11.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary8 2024-08-25T20:17:11.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary8 2024-08-25T20:17:11.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:17:11.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary8 2024-08-25T20:17:11.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:17:11.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:17:11.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:17:11.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:17:11.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 66382 2024-08-25T20:17:11.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 9 2024-08-25T20:17:11.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd8 2024-08-25T20:17:11.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary9 2024-08-25T20:17:11.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary9 2024-08-25T20:17:11.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:17:11.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary9 2024-08-25T20:17:11.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 10 2024-08-25T20:17:11.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd9 2024-08-25T20:17:11.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary10 2024-08-25T20:17:11.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary10 2024-08-25T20:17:11.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:17:11.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary10 2024-08-25T20:17:11.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:17:11.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:17:11.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:17:11.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:17:11.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:17:11.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:17:11.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:17:11.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:17:11.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:17:11.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:17:11.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:17:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:17:12.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:17:12.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:17:12.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:17:12.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:17:12.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:17:12.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:17:12.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:17:12.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:18:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=edfac7d467b1ffb523f82d3a633695d3 2024-08-25T20:18:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-08-25T20:18:03.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary6 2024-08-25T20:18:03.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:03.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:03.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:03.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-08-25T20:18:04.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:18:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-08-25T20:18:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:05.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:18:05.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:05.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:05.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:05.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:05.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:05.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:05.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:05.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:05.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:18:05.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:18:05.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0a0d8e9d-63de-44ab-bdb2-17b97792d7d0 2024-08-25T20:18:05.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:05.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:05.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:05.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:05.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:05.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:05.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:05.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:18:06.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=57e1203be07f765cf486470a42f92ec0 2024-08-25T20:18:06.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-08-25T20:18:06.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=6085eff896488d0f6fafed46f4afbcdb 2024-08-25T20:18:06.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-08-25T20:18:06.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary10 2024-08-25T20:18:06.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:06.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:06.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:18:06.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-08-25T20:18:06.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary7 2024-08-25T20:18:06.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:06.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:06.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:18:06.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-08-25T20:18:07.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=2a6e58574e49af42403d0111acb86d40 2024-08-25T20:18:07.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-08-25T20:18:07.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:18:07.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:18:07.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-08-25T20:18:07.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:07.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:18:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-08-25T20:18:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:18:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:07.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=b517d3a91b4b4d8ebd0b51d0d0a8d32f 2024-08-25T20:18:07.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-08-25T20:18:07.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary8 2024-08-25T20:18:07.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:07.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:07.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:18:07.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-08-25T20:18:07.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=cab3126440d5ce6f08bc3dab4e999c96 2024-08-25T20:18:07.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-08-25T20:18:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary5 2024-08-25T20:18:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:07.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:07.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:18:07.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-08-25T20:18:07.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=cb649c5d18780f1c72ef5f607f2be661 2024-08-25T20:18:07.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-08-25T20:18:07.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:18:07.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:07.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:07.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:07.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:07.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:07.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:07.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:18:07.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=cd5c57801e99d5b494d02043d7483c49 2024-08-25T20:18:07.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-08-25T20:18:07.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary1 2024-08-25T20:18:07.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:07.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:07.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:18:07.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-08-25T20:18:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:18:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0a0d8e9d-63de-44ab-bdb2-17b97792d7d0 2024-08-25T20:18:07.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:07.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:07.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:07.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:07.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:07.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:07.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:07.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:18:08.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary3 2024-08-25T20:18:08.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:08.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:08.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:18:08.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-08-25T20:18:08.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary4 2024-08-25T20:18:08.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:08.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:08.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:18:08.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-08-25T20:18:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=18cac91a8c665a3522232408df360640 2024-08-25T20:18:08.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-08-25T20:18:08.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary2 2024-08-25T20:18:08.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:08.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:08.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:18:08.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-08-25T20:18:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:18:08.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:18:08.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:18:08.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:08.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:08.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:18:08.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:08.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:08.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:08.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:08.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:08.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:18:08.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:18:08.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:08.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:08.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:18:08.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:08.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:08.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:08.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:08.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:08.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:18:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:18:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:18:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-08-25T20:18:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:18:08.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:08.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:08.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:08.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:08.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:08.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:18:08.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-08-25T20:18:08.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:08.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:08.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:18:08.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:08.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:08.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:08.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:08.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:08.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-08-25T20:18:08.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:08.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:08.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:18:08.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:08.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:08.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:08.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:08.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-08-25T20:18:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:18:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:08.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-08-25T20:18:08.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:08.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:08.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:18:08.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:08.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:08.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:08.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:08.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:08.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:18:08.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ce7df407-6dd2-4d59-8f98-8c52eaefd177 2024-08-25T20:18:08.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:08.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:08.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:08.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:08.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:08.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:08.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:18:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:18:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dda4aac1-b2d6-46d3-a987-574494886b5c 2024-08-25T20:18:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:08.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:08.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:08.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:08.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:08.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:18:08.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=2acd8cce03612c16c3c0bcf9f40b3257 2024-08-25T20:18:08.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-08-25T20:18:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary9 2024-08-25T20:18:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:18:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-08-25T20:18:09.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:18:09.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:18:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-08-25T20:18:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:18:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:09.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:09.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:09.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:18:09.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:09.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:09.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:18:09.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:09.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:09.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:09.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:09.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:09.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:18:09.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:18:09.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:09.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:09.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:18:09.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:09.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:09.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:09.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:09.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:09.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:18:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-08-25T20:18:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:18:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:09.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:18:09.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:09.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:09.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:18:09.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:09.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:09.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:09.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:18:09.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:18:09.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:09.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:18:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:09.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:09.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:18:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:18:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:18:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:09.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:09.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:18:09.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:18:09.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4 2024-08-25T20:18:09.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:09.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:09.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:09.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:09.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:09.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:09.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:09.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:18:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:18:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4f7fa68b-ea47-408d-8470-47214ea59e67 2024-08-25T20:18:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:18:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31496f9b-5900-44ec-b871-36538a41e65c 2024-08-25T20:18:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:18:09.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:09.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:09.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:18:09.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:18:09.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 54e7d5c4-e127-461b-8dd6-28ec2ebe7b7a 2024-08-25T20:18:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:18:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:18:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: cf4f7205-657c-4d10-848c-7bf5c56279da 2024-08-25T20:18:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:10.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:10.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:10.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:18:10.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:18:10.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:10.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:10.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:18:10.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:10.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:10.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:10.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:10.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:10.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:18:10.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:18:10.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:10.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:18:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:18:10.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:18:11.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 89d01a96-1a53-44bb-b8a5-c17d22e98de0 2024-08-25T20:18:11.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:11.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:11.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:11.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:11.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:18:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:18:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:18:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 975cc06f-b335-4181-8f15-5fd979f23a46 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ce7df407-6dd2-4d59-8f98-8c52eaefd177 2024-08-25T20:18:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dda4aac1-b2d6-46d3-a987-574494886b5c 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:18:11.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:18:11.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:11.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:11.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:18:11.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:11.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:11.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:11.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:11.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:11.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:18:11.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:18:11.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:11.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:11.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:18:11.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:11.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:11.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:11.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:11.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:11.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:18:11.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:18:11.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4 2024-08-25T20:18:11.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:11.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:11.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:11.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:11.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:11.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:11.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:11.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:18:11.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:18:11.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:11.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:11.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:11.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:11.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:11.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:11.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:11.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:11.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:18:11.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:18:11.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:11.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:11.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:18:11.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:11.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:11.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:11.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:11.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:11.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:18:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:18:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:18:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:18:12.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:18:12.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4f7fa68b-ea47-408d-8470-47214ea59e67 2024-08-25T20:18:12.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:12.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:12.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:12.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:18:12.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:18:12.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31496f9b-5900-44ec-b871-36538a41e65c 2024-08-25T20:18:12.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:12.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:12.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:12.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:12.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:18:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0a0d8e9d-63de-44ab-bdb2-17b97792d7d0 2024-08-25T20:18:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:12.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:12.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:12.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:12.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:12.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:12.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:12.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:18:12.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:12.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:12.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:12.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:18:12.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 54e7d5c4-e127-461b-8dd6-28ec2ebe7b7a 2024-08-25T20:18:12.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:12.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:12.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:12.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:12.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:12.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:12.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:12.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:18:12.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:18:12.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:12.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:12.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:18:12.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:12.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:12.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:12.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:12.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:12.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:18:12.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:18:12.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: cf4f7205-657c-4d10-848c-7bf5c56279da 2024-08-25T20:18:12.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:12.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:12.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:12.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:12.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:12.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:12.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:12.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:18:12.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:18:12.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:12.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:12.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:18:12.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:12.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:12.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:12.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:12.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:12.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:18:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:18:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:18:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:12.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:12.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:12.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:12.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:18:12.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:18:12.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 89d01a96-1a53-44bb-b8a5-c17d22e98de0 2024-08-25T20:18:12.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:12.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:12.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:12.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:12.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:12.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:12.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:12.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:18:12.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:18:12.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 975cc06f-b335-4181-8f15-5fd979f23a46 2024-08-25T20:18:12.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:12.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:12.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:12.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:12.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:12.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:12.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:12.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:18:14.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:18:14.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:18:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:18:14.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:18:14.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:14.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:14.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:18:14.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:18:14.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:18:14.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ce7df407-6dd2-4d59-8f98-8c52eaefd177 2024-08-25T20:18:14.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:14.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:14.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:18:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dda4aac1-b2d6-46d3-a987-574494886b5c 2024-08-25T20:18:14.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:15.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:15.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:18:15.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:15.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:15.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:18:15.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:15.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:15.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:15.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:15.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:15.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:18:16.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:18:16.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:16.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:16.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:18:16.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:16.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:16.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:16.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:16.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:16.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:18:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:18:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:18:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:16.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:16.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:18:16.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4 2024-08-25T20:18:16.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:16.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:16.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:16.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:16.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:16.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:16.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:18:16.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:16.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:16.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:16.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:18:16.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:16.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:16.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:18:16.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:16.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:16.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:16.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:16.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:16.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:18:16.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:18:16.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4f7fa68b-ea47-408d-8470-47214ea59e67 2024-08-25T20:18:16.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:16.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:16.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:16.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:16.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:16.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:18:16.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31496f9b-5900-44ec-b871-36538a41e65c 2024-08-25T20:18:16.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:16.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:16.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:16.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:16.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:16.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:16.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:16.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:16.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:18:16.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 54e7d5c4-e127-461b-8dd6-28ec2ebe7b7a 2024-08-25T20:18:16.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:16.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:16.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:16.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:16.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:16.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:16.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:16.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:16.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:18:16.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:16.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:16.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:18:16.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:16.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:16.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:16.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:16.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:16.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:18:16.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:18:16.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: cf4f7205-657c-4d10-848c-7bf5c56279da 2024-08-25T20:18:16.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:16.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:16.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:16.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:16.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:16.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:16.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:16.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:16.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:18:16.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:16.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:16.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:18:16.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:16.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:16.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:16.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:16.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:16.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:18:16.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:18:16.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:16.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:16.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:18:16.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:18:16.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:18:16.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 89d01a96-1a53-44bb-b8a5-c17d22e98de0 2024-08-25T20:18:16.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:16.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:16.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:16.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:16.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:16.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:18:16.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 975cc06f-b335-4181-8f15-5fd979f23a46 2024-08-25T20:18:16.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:16.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:16.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:16.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:16.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:18:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:20.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:20.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:18:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:18:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0a0d8e9d-63de-44ab-bdb2-17b97792d7d0 2024-08-25T20:18:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:20.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:20.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:20.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:23.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:18:23.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:23.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:23.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:18:23.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:23.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:23.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:23.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:23.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:23.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:18:23.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:18:23.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:23.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:23.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:18:23.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:23.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:23.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:23.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:23.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:23.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:18:23.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:18:23.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ce7df407-6dd2-4d59-8f98-8c52eaefd177 2024-08-25T20:18:23.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:23.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:23.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:23.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:23.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:23.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:18:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dda4aac1-b2d6-46d3-a987-574494886b5c 2024-08-25T20:18:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:23.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:23.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:18:24.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:24.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:24.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:18:24.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:24.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:24.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:24.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:24.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:24.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:18:24.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:18:24.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:24.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:24.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:18:24.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:24.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:24.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:24.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:24.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:24.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:18:24.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:18:24.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:24.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:24.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:18:24.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:24.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:24.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:24.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:24.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:24.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:18:24.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:18:24.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4 2024-08-25T20:18:24.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:24.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:24.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:24.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:24.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:24.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:24.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:24.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:24.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:18:24.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:24.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:24.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:18:24.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:24.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:24.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:24.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:24.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:24.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:18:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:18:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4f7fa68b-ea47-408d-8470-47214ea59e67 2024-08-25T20:18:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:24.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:24.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:24.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:24.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:18:24.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31496f9b-5900-44ec-b871-36538a41e65c 2024-08-25T20:18:24.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:24.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:24.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:24.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:24.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:24.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:24.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:24.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:24.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:18:24.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:24.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:24.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:18:24.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:24.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:24.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:24.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:24.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:24.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:18:24.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:18:24.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 54e7d5c4-e127-461b-8dd6-28ec2ebe7b7a 2024-08-25T20:18:24.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:24.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:24.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:24.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:24.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:18:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: cf4f7205-657c-4d10-848c-7bf5c56279da 2024-08-25T20:18:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:24.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:24.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:24.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:24.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:18:24.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:24.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:24.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:18:24.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:24.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:24.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:24.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:24.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:24.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:18:24.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:18:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:18:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:24.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:18:25.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:18:25.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 89d01a96-1a53-44bb-b8a5-c17d22e98de0 2024-08-25T20:18:25.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:25.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:25.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:25.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:25.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:25.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:25.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:25.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:18:25.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 975cc06f-b335-4181-8f15-5fd979f23a46 2024-08-25T20:18:25.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:25.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:25.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:25.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:25.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:18:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:28.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:28.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:18:28.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:18:28.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0a0d8e9d-63de-44ab-bdb2-17b97792d7d0 2024-08-25T20:18:28.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:18:28.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:18:28.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:28.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:17:58' 2024-08-25T20:18:28.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:28.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:18:28.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:28.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:18:31.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:18:31.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:31.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:31.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:18:31.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:31.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:31.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:31.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:31.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:31.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:18:31.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:18:31.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:31.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:31.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:18:31.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:31.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:31.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:31.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:31.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:31.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:18:31.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:18:31.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ce7df407-6dd2-4d59-8f98-8c52eaefd177 2024-08-25T20:18:31.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:31.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:31.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:31.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:31.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:31.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:18:31.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dda4aac1-b2d6-46d3-a987-574494886b5c 2024-08-25T20:18:31.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:31.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:31.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:31.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:31.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:31.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:31.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:31.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:31.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:31.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:31.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:31.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:31.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:31.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:31.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:31.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:31.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:31.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:31.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:31.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:31.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:31.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:31.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:31.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:31.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:31.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:31.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:31.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:31.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:31.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:31.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:31.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:31.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:31.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:31.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:31.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:31.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:18:31.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:31.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:31.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:18:31.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:31.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:31.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:18:31.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:18:31.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:31.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:31.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:31.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:31.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:31.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:31.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:31.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-08-25T20:18:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:18:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:31.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:32.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:18:32.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:32.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:18:32.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:32.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:32.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:18:32.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:18:32.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:18:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:18:32.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:18:32.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:32.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:18:32.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:32.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:32.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:18:32.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:18:32.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4 2024-08-25T20:18:32.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:32.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:32.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:32.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:32.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:32.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:32.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:32.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:18:32.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:32.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:18:32.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:32.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:32.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:18:32.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:32.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:32.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:32.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:32.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:32.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:32.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:18:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:32.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:32.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:18:32.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:32.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:18:32.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:32.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:32.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:18:32.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-08-25T20:18:32.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:32.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:32.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-08-25T20:18:32.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-08-25T20:18:32.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:18:32.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ce7df407-6dd2-4d59-8f98-8c52eaefd177 2024-08-25T20:18:32.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:32.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:32.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:32.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:32.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:32.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:18:32.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dda4aac1-b2d6-46d3-a987-574494886b5c 2024-08-25T20:18:32.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:32.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:32.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:32.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:32.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:32.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:32.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:32.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:32.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:32.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:32.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:32.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:32.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:32.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:32.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:32.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:32.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:32.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:32.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:32.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:18:32.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4f7fa68b-ea47-408d-8470-47214ea59e67 2024-08-25T20:18:32.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:32.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:32.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:32.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:32.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:32.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:32.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:32.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:32.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:32.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:18:32.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31496f9b-5900-44ec-b871-36538a41e65c 2024-08-25T20:18:32.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:32.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:32.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:32.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:32.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:32.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:32.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:32.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:32.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:32.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:18:32.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:32.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:32.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:32.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:32.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:32.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:18:32.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:32.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:32.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:32.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:32.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:32.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:18:32.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: cf4f7205-657c-4d10-848c-7bf5c56279da 2024-08-25T20:18:32.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:32.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:32.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:32.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:32.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:32.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:32.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:32.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:32.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:18:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 54e7d5c4-e127-461b-8dd6-28ec2ebe7b7a 2024-08-25T20:18:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:32.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:32.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:32.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:32.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:32.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:32.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:32.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:32.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:32.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:32.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:32.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:32.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:18:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:32.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:32.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:32.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:32.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:32.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:32.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:18:32.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:32.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:32.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:32.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:32.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:32.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:32.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:32.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:32.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:32.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:18:32.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:32.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:32.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:32.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:32.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:18:32.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:32.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:32.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:32.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:32.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:32.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:32.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:18:32.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:18:32.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:32.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:32.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-08-25T20:18:32.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:32.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:18:32.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:32.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:32.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:32.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:32.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:18:32.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:32.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:32.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:18:32.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:18:32.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:18:32.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:32.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary10 2024-08-25T20:18:32.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:32.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:18:32.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:18:32.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-08-25T20:18:32.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary7 2024-08-25T20:18:32.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:32.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:18:32.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:18:32.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-08-25T20:18:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:32.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:18:32.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:18:32.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:32.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:32.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:18:32.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:32.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:18:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-08-25T20:18:32.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:32.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:18:32.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:32.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:32.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:32.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:32.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:32.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:32.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:18:32.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:32.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:32.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:32.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:32.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:18:32.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:18:32.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-08-25T20:18:32.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:32.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:18:32.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:32.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:32.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:32.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:32.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:32.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:18:32.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:32.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:32.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:32.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-08-25T20:18:32.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:32.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:18:32.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:32.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:32.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:32.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:32.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:32.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:32.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:32.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-08-25T20:18:32.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:32.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:32.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:18:32.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:32.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:32.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:32.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:32.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:33.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:18:33.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:33.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:18:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:18:33.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:18:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:18:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:18:33.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:18:33.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 89d01a96-1a53-44bb-b8a5-c17d22e98de0 2024-08-25T20:18:33.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:33.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:33.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:33.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:33.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:33.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:33.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:33.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:33.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:33.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:33.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:33.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:33.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:33.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:33.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:33.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:18:33.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:33.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:18:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 975cc06f-b335-4181-8f15-5fd979f23a46 2024-08-25T20:18:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:33.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:33.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:33.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:33.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:33.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:33.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:33.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:33.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:33.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:33.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:33.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:18:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:33.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:18:33.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:18:33.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:18:33.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:18:33.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-08-25T20:18:33.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:18:33.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:18:33.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-08-25T20:18:33.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:33.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:33.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:18:33.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:33.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:18:33.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:33.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:33.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:33.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-08-25T20:18:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:18:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:33.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:18:33.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:33.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:18:33.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:33.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:33.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-08-25T20:18:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:18:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:33.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:33.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd0 2024-08-25T20:18:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd1 2024-08-25T20:18:33.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:18:33.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:18:33.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:18:33.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:18:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-08-25T20:18:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:18:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-08-25T20:18:33.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:18:33.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4 2024-08-25T20:18:33.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:33.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:33.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:33.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:33.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:33.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:33.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:33.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:33.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:33.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:33.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:33.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:33.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:33.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:33.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:33.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:18:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:33.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-08-25T20:18:33.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:18:33.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:33.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:33.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-08-25T20:18:33.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-08-25T20:18:33.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:18:33.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:33.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:33.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-08-25T20:18:33.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-08-25T20:18:33.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:18:33.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:33.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:33.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-08-25T20:18:33.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-08-25T20:18:33.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:18:33.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:33.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:33.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-08-25T20:18:33.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:33.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:33.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:18:33.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:33.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:18:33.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:33.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:33.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:33.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:33.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:33.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:33.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary1 2024-08-25T20:18:33.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:18:33.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:18:33.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-08-25T20:18:33.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:18:33.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 54e7d5c4-e127-461b-8dd6-28ec2ebe7b7a 2024-08-25T20:18:33.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:33.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:33.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:33.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:33.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:33.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:33.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:33.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:33.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:33.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:33.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:33.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:33.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:33.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:18:33.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: cf4f7205-657c-4d10-848c-7bf5c56279da 2024-08-25T20:18:33.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:33.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:33.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:33.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:33.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:33.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:33.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:33.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:18:33.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31496f9b-5900-44ec-b871-36538a41e65c 2024-08-25T20:18:33.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:33.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:33.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:33.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:33.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:33.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:33.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:33.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:33.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:33.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:33.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:33.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:33.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:33.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:18:33.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:33.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:33.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:33.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:33.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:33.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:33.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:33.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:33.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:33.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:33.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:18:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4f7fa68b-ea47-408d-8470-47214ea59e67 2024-08-25T20:18:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:33.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:33.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:33.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:33.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:33.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:33.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:33.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:33.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:18:33.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:33.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:33.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:33.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:33.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:33.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:33.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:33.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:18:33.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:33.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:33.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:33.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:18:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:33.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:33.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:33.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:34.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:34.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:34.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:18:34.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:34.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:18:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:34.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:34.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:34.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:34.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:34.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:18:34.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:34.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:34.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:34.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary5 2024-08-25T20:18:34.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:34.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:34.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:18:34.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:18:34.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-08-25T20:18:34.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:34.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:18:34.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:34.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:34.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:34.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:34.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary8 2024-08-25T20:18:34.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:34.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:34.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:18:34.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:18:34.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-08-25T20:18:34.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:34.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:34.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:18:34.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:18:34.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:34.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:18:34.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:34.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:34.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:34.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:34.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary3 2024-08-25T20:18:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:18:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:18:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-08-25T20:18:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:18:34.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:18:34.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:34.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:34.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:34.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:34.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:34.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary4 2024-08-25T20:18:34.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:34.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:34.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:18:34.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:18:34.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-08-25T20:18:34.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:18:34.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:18:34.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-08-25T20:18:34.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-08-25T20:18:34.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:34.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:34.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:18:34.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:34.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:34.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:34.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:34.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:34.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-08-25T20:18:34.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-08-25T20:18:34.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:34.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:34.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:18:34.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:34.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:34.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:34.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-08-25T20:18:34.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:18:34.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 89d01a96-1a53-44bb-b8a5-c17d22e98de0 2024-08-25T20:18:34.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:34.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:34.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:34.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:34.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:34.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:34.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:34.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:34.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:34.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:34.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:34.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:34.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:34.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:34.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:34.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:34.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:34.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:18:34.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:34.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:34.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:34.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:34.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:34.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd2 2024-08-25T20:18:34.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:18:34.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:18:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:18:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 975cc06f-b335-4181-8f15-5fd979f23a46 2024-08-25T20:18:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:34.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:34.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:34.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:34.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:34.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:34.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:34.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:34.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:34.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:34.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:18:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:34.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:34.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:34.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:34.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:18:34.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:34.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:18:34.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:34.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:34.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:34.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:34.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:34.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:34.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary2 2024-08-25T20:18:34.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:34.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:34.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:18:34.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:18:34.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-08-25T20:18:34.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:34.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:34.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:18:34.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:34.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:18:34.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:34.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:34.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:34.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:34.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:34.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:34.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary9 2024-08-25T20:18:34.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:34.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:34.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:18:34.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:18:34.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-08-25T20:18:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:18:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:18:35.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:18:35.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:18:35.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:18:35.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:18:35.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:18:35.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:18:35.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-08-25T20:18:35.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:18:35.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:18:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-08-25T20:18:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:18:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:18:35.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-08-25T20:18:35.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:18:35.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:18:35.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-08-25T20:18:35.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:18:35.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:18:35.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-08-25T20:18:35.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:18:35.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:18:35.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-08-25T20:18:35.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd4 2024-08-25T20:18:35.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:18:35.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:18:35.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd3 2024-08-25T20:18:35.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd5 2024-08-25T20:18:35.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:18:35.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:18:35.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:18:35.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:18:35.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd7 2024-08-25T20:18:35.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd6 2024-08-25T20:18:35.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:18:35.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:18:35.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:18:35.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:18:35.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd8 2024-08-25T20:18:35.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:18:35.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:18:36.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:18:36.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:36.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:36.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:36.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:36.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:36.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:36.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:36.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:36.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:18:36.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:18:36.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0a0d8e9d-63de-44ab-bdb2-17b97792d7d0 2024-08-25T20:18:36.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:36.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:36.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:36.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:36.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:36.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:36.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:36.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:36.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:36.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:36.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:36.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:36.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:36.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:36.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:18:36.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:36.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:36.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:36.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:36.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:36.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:36.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:36.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:36.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:36.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:18:36.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:18:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:36.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:36.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:36.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:36.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:36.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:36.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-08-25T20:18:36.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:36.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:36.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:36.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:36.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:36.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:36.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:18:36.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:18:37.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-08-25T20:18:37.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:37.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:37.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:37.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:37.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:18:37.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:18:37.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:37.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:37.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-08-25T20:18:38.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:18:38.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0a0d8e9d-63de-44ab-bdb2-17b97792d7d0 2024-08-25T20:18:38.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:18:38.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:18:38.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:18:38.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:18:28' 2024-08-25T20:18:38.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:18:38.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:18:38.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:18:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:18:38.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:18:38.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:38.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:18:38.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:18:38.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:18:38.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:18:38.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:18:38.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-08-25 20:18:28' 2024-08-25T20:18:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:18:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:18:28' 2024-08-25T20:18:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:18:38.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:18:38.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:18:38.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:18:28' 2024-08-25T20:18:38.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:18:38.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:18:38.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:18:28' 2024-08-25T20:18:38.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:18:38.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:18:38.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:18:38.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:18:38.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:18:38.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:38.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:18:38.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary6 2024-08-25T20:18:38.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:18:38.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:18:38.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:18:38.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:18:38.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-08-25T20:18:39.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:18:39.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:18:39.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:18:39.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-08-25T20:18:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd9 2024-08-25T20:18:39.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:18:39.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:19:26.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=57e1203be07f765cf486470a42f92ec0 2024-08-25T20:19:26.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-08-25T20:19:27.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 57e1203be07f765cf486470a42f92ec0 != \5\7\e\1\2\0\3\b\e\0\7\f\7\6\5\c\f\4\8\6\4\7\0\a\4\2\f\9\2\e\c\0 ]] 2024-08-25T20:19:27.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=6085eff896488d0f6fafed46f4afbcdb 2024-08-25T20:19:27.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-08-25T20:19:27.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 6085eff896488d0f6fafed46f4afbcdb != \6\0\8\5\e\f\f\8\9\6\4\8\8\d\0\f\6\f\a\f\e\d\4\6\f\4\a\f\b\c\d\b ]] 2024-08-25T20:19:28.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=cb649c5d18780f1c72ef5f607f2be661 2024-08-25T20:19:28.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-08-25T20:19:28.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ cb649c5d18780f1c72ef5f607f2be661 != \c\b\6\4\9\c\5\d\1\8\7\8\0\f\1\c\7\2\e\f\5\f\6\0\7\f\2\b\e\6\6\1 ]] 2024-08-25T20:19:29.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=cab3126440d5ce6f08bc3dab4e999c96 2024-08-25T20:19:29.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-08-25T20:19:29.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ cab3126440d5ce6f08bc3dab4e999c96 != \c\a\b\3\1\2\6\4\4\0\d\5\c\e\6\f\0\8\b\c\3\d\a\b\4\e\9\9\9\c\9\6 ]] 2024-08-25T20:19:29.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:19:29.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 66383 2024-08-25T20:19:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=2acd8cce03612c16c3c0bcf9f40b3257 2024-08-25T20:19:29.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-08-25T20:19:29.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 2acd8cce03612c16c3c0bcf9f40b3257 != \2\a\c\d\8\c\c\e\0\3\6\1\2\c\1\6\c\3\c\0\b\c\f\9\f\4\0\b\3\2\5\7 ]] 2024-08-25T20:19:30.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=cd5c57801e99d5b494d02043d7483c49 2024-08-25T20:19:30.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-08-25T20:19:30.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=18cac91a8c665a3522232408df360640 2024-08-25T20:19:30.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-08-25T20:19:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=b517d3a91b4b4d8ebd0b51d0d0a8d32f 2024-08-25T20:19:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-08-25T20:19:30.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ cd5c57801e99d5b494d02043d7483c49 != \c\d\5\c\5\7\8\0\1\e\9\9\d\5\b\4\9\4\d\0\2\0\4\3\d\7\4\8\3\c\4\9 ]] 2024-08-25T20:19:30.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 18cac91a8c665a3522232408df360640 != \1\8\c\a\c\9\1\a\8\c\6\6\5\a\3\5\2\2\2\3\2\4\0\8\d\f\3\6\0\6\4\0 ]] 2024-08-25T20:19:30.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:19:30.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 66385 2024-08-25T20:19:30.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:19:30.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 66387 2024-08-25T20:19:30.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:19:30.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 66389 2024-08-25T20:19:30.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ b517d3a91b4b4d8ebd0b51d0d0a8d32f != \b\5\1\7\d\3\a\9\1\b\4\b\4\d\8\e\b\d\0\b\5\1\d\0\d\0\a\8\d\3\2\f ]] 2024-08-25T20:19:30.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:19:30.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 66391 2024-08-25T20:19:30.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=2a6e58574e49af42403d0111acb86d40 2024-08-25T20:19:30.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-08-25T20:19:30.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 2a6e58574e49af42403d0111acb86d40 != \2\a\6\e\5\8\5\7\4\e\4\9\a\f\4\2\4\0\3\d\0\1\1\1\a\c\b\8\6\d\4\0 ]] 2024-08-25T20:19:31.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=edfac7d467b1ffb523f82d3a633695d3 2024-08-25T20:19:31.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-08-25T20:19:31.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ edfac7d467b1ffb523f82d3a633695d3 != \e\d\f\a\c\7\d\4\6\7\b\1\f\f\b\5\2\3\f\8\2\d\3\a\6\3\3\6\9\5\d\3 ]] 2024-08-25T20:19:31.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:19:31.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 66393 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 66395 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 66397 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 66399 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:19:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:19:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-08-25T20:19:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:19:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:19:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-08-25T20:19:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:19:33.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:33.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:33.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:33.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:33.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:33.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:19:33.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:19:33.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4 2024-08-25T20:19:33.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:19:33.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3327385.6,"bytes_per_snapshot":66547712.0,"last_snapshot_bytes":66547712,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617114,"remote_snapshot_timestamp":1724617114,"replay_state":"idle"} 2024-08-25T20:19:33.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:19:33.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:19:28' 2024-08-25T20:19:33.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:19:33.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:19:33.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:19:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3327385.6,"bytes_per_snapshot":66547712.0,"last_snapshot_bytes":66547712,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617114,"remote_snapshot_timestamp":1724617114,"replay_state":"idle"} 2024-08-25T20:19:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:19:33.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:33.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:19:33.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:33.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:19:33.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:19:33.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:19:33.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:19:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-08-25 20:19:28' 2024-08-25T20:19:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:19:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:19:28' 2024-08-25T20:19:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:33.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:19:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary15bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4up+replayingreplaying, {"bytes_per_second":3327385.6,"bytes_per_snapshot":66547712.0,"last_snapshot_bytes":66547712,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617114,"remote_snapshot_timestamp":1724617114,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+replayingreplaying, {"bytes_per_second":3241369.6,"bytes_per_snapshot":64827392.0,"last_snapshot_bytes":64827392,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+replayingreplaying, {"bytes_per_second":3229081.6,"bytes_per_snapshot":64581632.0,"last_snapshot_bytes":64581632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+replayingreplaying, {"bytes_per_second":3354419.2,"bytes_per_snapshot":67088384.0,"last_snapshot_bytes":67088384,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+replayingreplaying, {"bytes_per_second":3498393.6,"bytes_per_snapshot":69967872.0,"last_snapshot_bytes":69967872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+replayingreplaying, {"bytes_per_second":3406028.8,"bytes_per_snapshot":68120576.0,"last_snapshot_bytes":68120576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+replayingreplaying, {"bytes_per_second":3242598.4,"bytes_per_snapshot":64851968.0,"last_snapshot_bytes":64851968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617119,"remote_snapshot_timestamp":1724617119,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+replayingreplaying, {"bytes_per_second":3491840.0,"bytes_per_snapshot":69836800.0,"last_snapshot_bytes":69836800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+replayingreplaying, {"bytes_per_second":3236659.2,"bytes_per_snapshot":64733184.0,"last_snapshot_bytes":64733184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+replayingreplaying, {"bytes_per_second":3211673.6,"bytes_per_snapshot":64233472.0,"last_snapshot_bytes":64233472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28' 2024-08-25T20:19:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:19:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:19:33.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:19:33.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:19:33.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:33.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:19:33.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:33.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:33.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:33.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary1 2024-08-25T20:19:33.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:19:33.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:33.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:19:33.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-08-25T20:19:33.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-08-25T20:19:33.606 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:33.606+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8ac9c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:19:33.607 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:33.606+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972f00 [3/5bde5fe9-f0a4-4b99-96bb-c6330fa6c4f4] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:19:35.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:19:35.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-08-25T20:19:35.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:35.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:35.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:19:35.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:35.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:19:35.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary1 2024-08-25T20:19:35.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:35.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary1 2024-08-25T20:19:35.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:19:35.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:19:35.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-08-25T20:19:35.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:35.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:35.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:19:35.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:19:35.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-08-25T20:19:35.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:35.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:35.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:19:35.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:35.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:35.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:35.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:35.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:19:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-08-25T20:19:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:19:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:19:36.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:19:36.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 89d01a96-1a53-44bb-b8a5-c17d22e98de0 2024-08-25T20:19:36.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:19:36.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3229081.6,"bytes_per_snapshot":64581632.0,"last_snapshot_bytes":64581632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"} 2024-08-25T20:19:36.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:19:36.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:19:28' 2024-08-25T20:19:36.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:19:36.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:19:36.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:19:36.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3229081.6,"bytes_per_snapshot":64581632.0,"last_snapshot_bytes":64581632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"} 2024-08-25T20:19:36.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:19:36.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:36.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:19:36.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:36.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:19:36.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:19:36.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:19:36.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:19:36.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:36.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-08-25 20:19:28' 2024-08-25T20:19:36.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:36.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:36.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:19:36.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:36.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:19:28' 2024-08-25T20:19:36.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:36.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:36.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:19:36.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING941904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+replayingreplaying, {"bytes_per_second":3241369.6,"bytes_per_snapshot":64827392.0,"last_snapshot_bytes":64827392,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary289d01a96-1a53-44bb-b8a5-c17d22e98de0up+replayingreplaying, {"bytes_per_second":3229081.6,"bytes_per_snapshot":64581632.0,"last_snapshot_bytes":64581632,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+replayingreplaying, {"bytes_per_second":3354419.2,"bytes_per_snapshot":67088384.0,"last_snapshot_bytes":67088384,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+replayingreplaying, {"bytes_per_second":3498393.6,"bytes_per_snapshot":69967872.0,"last_snapshot_bytes":69967872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+replayingreplaying, {"bytes_per_second":3406028.8,"bytes_per_snapshot":68120576.0,"last_snapshot_bytes":68120576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+replayingreplaying, {"bytes_per_second":3242598.4,"bytes_per_snapshot":64851968.0,"last_snapshot_bytes":64851968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617119,"remote_snapshot_timestamp":1724617119,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+replayingreplaying, {"bytes_per_second":3491840.0,"bytes_per_snapshot":69836800.0,"last_snapshot_bytes":69836800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+replayingreplaying, {"bytes_per_second":3236659.2,"bytes_per_snapshot":64733184.0,"last_snapshot_bytes":64733184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+replayingreplaying, {"bytes_per_second":3211673.6,"bytes_per_snapshot":64233472.0,"last_snapshot_bytes":64233472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28' 2024-08-25T20:19:36.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:19:36.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:19:36.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:36.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:19:36.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:19:36.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:36.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:19:36.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:36.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:36.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:36.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary2 2024-08-25T20:19:36.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:19:36.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:36.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:19:36.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-08-25T20:19:36.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-08-25T20:19:36.905 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:36.904+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3330d00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:19:36.905 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:36.904+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973b80 [3/89d01a96-1a53-44bb-b8a5-c17d22e98de0] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:19:38.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:19:38.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-08-25T20:19:38.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:38.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:38.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:19:38.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:38.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:19:38.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary2 2024-08-25T20:19:38.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:38.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary2 2024-08-25T20:19:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:19:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:19:39.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-08-25T20:19:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:19:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:19:39.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:19:39.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31496f9b-5900-44ec-b871-36538a41e65c 2024-08-25T20:19:39.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:19:39.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3354419.2,"bytes_per_snapshot":67088384.0,"last_snapshot_bytes":67088384,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"} 2024-08-25T20:19:39.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:19:39.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:19:28' 2024-08-25T20:19:39.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:19:39.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:19:39.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:19:39.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3354419.2,"bytes_per_snapshot":67088384.0,"last_snapshot_bytes":67088384,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"} 2024-08-25T20:19:39.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:19:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:19:39.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:39.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:19:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:19:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:19:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:19:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-08-25 20:19:28' 2024-08-25T20:19:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:19:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:19:28' 2024-08-25T20:19:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:19:39.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING841904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+replayingreplaying, {"bytes_per_second":3241369.6,"bytes_per_snapshot":64827392.0,"last_snapshot_bytes":64827392,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary331496f9b-5900-44ec-b871-36538a41e65cup+replayingreplaying, {"bytes_per_second":3354419.2,"bytes_per_snapshot":67088384.0,"last_snapshot_bytes":67088384,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+replayingreplaying, {"bytes_per_second":3498393.6,"bytes_per_snapshot":69967872.0,"last_snapshot_bytes":69967872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+replayingreplaying, {"bytes_per_second":3406028.8,"bytes_per_snapshot":68120576.0,"last_snapshot_bytes":68120576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+replayingreplaying, {"bytes_per_second":3242598.4,"bytes_per_snapshot":64851968.0,"last_snapshot_bytes":64851968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617119,"remote_snapshot_timestamp":1724617119,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+replayingreplaying, {"bytes_per_second":3491840.0,"bytes_per_snapshot":69836800.0,"last_snapshot_bytes":69836800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+replayingreplaying, {"bytes_per_second":3236659.2,"bytes_per_snapshot":64733184.0,"last_snapshot_bytes":64733184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+replayingreplaying, {"bytes_per_second":3211673.6,"bytes_per_snapshot":64233472.0,"last_snapshot_bytes":64233472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28' 2024-08-25T20:19:39.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:19:39.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:19:39.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:39.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:19:39.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:19:39.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:39.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:19:39.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:39.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:39.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:39.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary3 2024-08-25T20:19:39.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:19:39.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:39.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:19:39.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-08-25T20:19:39.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-08-25T20:19:39.910 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:39.910+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae34544e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:19:39.910 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:39.910+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972780 [3/31496f9b-5900-44ec-b871-36538a41e65c] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:19:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:19:41.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-08-25T20:19:41.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:41.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:41.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:19:41.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:41.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:19:41.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary3 2024-08-25T20:19:41.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:41.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary3 2024-08-25T20:19:41.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:19:41.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:19:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-08-25T20:19:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:19:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:19:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-08-25T20:19:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:19:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:41.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:41.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:41.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:19:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-08-25T20:19:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:19:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:42.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:42.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:42.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:19:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:19:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4f7fa68b-ea47-408d-8470-47214ea59e67 2024-08-25T20:19:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:19:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3498393.6,"bytes_per_snapshot":69967872.0,"last_snapshot_bytes":69967872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"} 2024-08-25T20:19:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:19:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:19:28' 2024-08-25T20:19:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:19:42.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:19:42.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:19:42.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3498393.6,"bytes_per_snapshot":69967872.0,"last_snapshot_bytes":69967872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"} 2024-08-25T20:19:42.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:19:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:42.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:19:42.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:42.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:19:42.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:19:42.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:19:42.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:19:42.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:42.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-08-25 20:19:28' 2024-08-25T20:19:42.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:42.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:42.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:19:42.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:42.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:19:28' 2024-08-25T20:19:42.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:42.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:42.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:19:42.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING741904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+replayingreplaying, {"bytes_per_second":3241369.6,"bytes_per_snapshot":64827392.0,"last_snapshot_bytes":64827392,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary44f7fa68b-ea47-408d-8470-47214ea59e67up+replayingreplaying, {"bytes_per_second":3498393.6,"bytes_per_snapshot":69967872.0,"last_snapshot_bytes":69967872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+replayingreplaying, {"bytes_per_second":3406028.8,"bytes_per_snapshot":68120576.0,"last_snapshot_bytes":68120576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+replayingreplaying, {"bytes_per_second":3242598.4,"bytes_per_snapshot":64851968.0,"last_snapshot_bytes":64851968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617119,"remote_snapshot_timestamp":1724617119,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+replayingreplaying, {"bytes_per_second":3491840.0,"bytes_per_snapshot":69836800.0,"last_snapshot_bytes":69836800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+replayingreplaying, {"bytes_per_second":3236659.2,"bytes_per_snapshot":64733184.0,"last_snapshot_bytes":64733184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+replayingreplaying, {"bytes_per_second":3211673.6,"bytes_per_snapshot":64233472.0,"last_snapshot_bytes":64233472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28' 2024-08-25T20:19:42.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:19:42.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:19:42.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:42.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:19:42.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:19:42.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:42.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:19:42.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:42.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:42.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:42.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary4 2024-08-25T20:19:42.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:19:42.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:42.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:19:42.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-08-25T20:19:42.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-08-25T20:19:42.768 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:42.767+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3455a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:19:42.768 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:42.767+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972c80 [3/4f7fa68b-ea47-408d-8470-47214ea59e67] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:19:44.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:19:44.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-08-25T20:19:44.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:44.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:44.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:19:44.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:44.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:19:44.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary4 2024-08-25T20:19:44.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:44.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary4 2024-08-25T20:19:44.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:19:44.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:19:44.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-08-25T20:19:44.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:44.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:44.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:19:44.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:19:44.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-08-25T20:19:44.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:44.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:44.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:19:44.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:44.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:44.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:44.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:44.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:19:45.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-08-25T20:19:45.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:45.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:45.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:19:45.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:45.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:45.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:45.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:45.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:45.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:19:45.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:19:45.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 54e7d5c4-e127-461b-8dd6-28ec2ebe7b7a 2024-08-25T20:19:45.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:19:45.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3406028.8,"bytes_per_snapshot":68120576.0,"last_snapshot_bytes":68120576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"} 2024-08-25T20:19:45.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:19:45.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:19:28' 2024-08-25T20:19:45.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:19:45.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:19:45.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:19:45.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3406028.8,"bytes_per_snapshot":68120576.0,"last_snapshot_bytes":68120576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"} 2024-08-25T20:19:45.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:19:45.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:45.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:19:45.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:45.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:19:45.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:19:45.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:19:45.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:19:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-08-25 20:19:28' 2024-08-25T20:19:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:19:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:19:28' 2024-08-25T20:19:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:45.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:45.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:19:45.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING641904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+replayingreplaying, {"bytes_per_second":3241369.6,"bytes_per_snapshot":64827392.0,"last_snapshot_bytes":64827392,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary554e7d5c4-e127-461b-8dd6-28ec2ebe7b7aup+replayingreplaying, {"bytes_per_second":3406028.8,"bytes_per_snapshot":68120576.0,"last_snapshot_bytes":68120576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+replayingreplaying, {"bytes_per_second":3242598.4,"bytes_per_snapshot":64851968.0,"last_snapshot_bytes":64851968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617119,"remote_snapshot_timestamp":1724617119,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+replayingreplaying, {"bytes_per_second":3491840.0,"bytes_per_snapshot":69836800.0,"last_snapshot_bytes":69836800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+replayingreplaying, {"bytes_per_second":3236659.2,"bytes_per_snapshot":64733184.0,"last_snapshot_bytes":64733184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+replayingreplaying, {"bytes_per_second":3211673.6,"bytes_per_snapshot":64233472.0,"last_snapshot_bytes":64233472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28' 2024-08-25T20:19:45.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:19:45.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:19:45.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:45.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:19:45.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:19:45.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:45.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:19:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary5 2024-08-25T20:19:45.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:19:45.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:45.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:19:45.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-08-25T20:19:45.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-08-25T20:19:45.770 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:45.769+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4fca1a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:19:45.770 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:45.769+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972280 [3/54e7d5c4-e127-461b-8dd6-28ec2ebe7b7a] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:19:47.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:19:47.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-08-25T20:19:47.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:47.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:47.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:19:47.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:47.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:19:47.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary5 2024-08-25T20:19:47.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:47.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary5 2024-08-25T20:19:47.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:19:47.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:19:47.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-08-25T20:19:47.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:19:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:19:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-08-25T20:19:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:19:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:19:48.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-08-25T20:19:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:19:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:48.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:48.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:19:48.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:19:48.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0a0d8e9d-63de-44ab-bdb2-17b97792d7d0 2024-08-25T20:19:48.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:19:48.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3242598.4,"bytes_per_snapshot":64851968.0,"last_snapshot_bytes":64851968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617119,"remote_snapshot_timestamp":1724617119,"replay_state":"idle"} 2024-08-25T20:19:48.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:19:48.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:19:28' 2024-08-25T20:19:48.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:19:48.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:19:48.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:19:48.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3242598.4,"bytes_per_snapshot":64851968.0,"last_snapshot_bytes":64851968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617119,"remote_snapshot_timestamp":1724617119,"replay_state":"idle"} 2024-08-25T20:19:48.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:19:48.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:48.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:19:48.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:48.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:19:48.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:19:48.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:19:48.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:19:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-08-25 20:19:28' 2024-08-25T20:19:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:19:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:48.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:19:28' 2024-08-25T20:19:48.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:48.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:48.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:19:48.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING541904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+replayingreplaying, {"bytes_per_second":3241369.6,"bytes_per_snapshot":64827392.0,"last_snapshot_bytes":64827392,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary60a0d8e9d-63de-44ab-bdb2-17b97792d7d0up+replayingreplaying, {"bytes_per_second":3242598.4,"bytes_per_snapshot":64851968.0,"last_snapshot_bytes":64851968,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617119,"remote_snapshot_timestamp":1724617119,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+replayingreplaying, {"bytes_per_second":3491840.0,"bytes_per_snapshot":69836800.0,"last_snapshot_bytes":69836800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+replayingreplaying, {"bytes_per_second":3236659.2,"bytes_per_snapshot":64733184.0,"last_snapshot_bytes":64733184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+replayingreplaying, {"bytes_per_second":3211673.6,"bytes_per_snapshot":64233472.0,"last_snapshot_bytes":64233472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28' 2024-08-25T20:19:48.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:19:48.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:19:48.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:48.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:19:48.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:19:48.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:48.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:19:48.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:48.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:48.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:48.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary6 2024-08-25T20:19:48.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:19:48.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:48.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:19:48.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-08-25T20:19:48.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-08-25T20:19:48.915 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:48.915+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade2d71e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:19:48.916 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:48.915+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ae4fe2a00 [3/0a0d8e9d-63de-44ab-bdb2-17b97792d7d0] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:19:50.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:19:50.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-08-25T20:19:50.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:50.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:50.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:19:50.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:50.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:19:50.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary6 2024-08-25T20:19:50.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:50.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary6 2024-08-25T20:19:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:19:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:19:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-08-25T20:19:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:19:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:19:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-08-25T20:19:50.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:50.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:50.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:19:50.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:50.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:50.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:50.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:50.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:19:51.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-08-25T20:19:51.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:51.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:51.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:19:51.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:51.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:51.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:51.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:51.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:51.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:19:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:19:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ce7df407-6dd2-4d59-8f98-8c52eaefd177 2024-08-25T20:19:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:19:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3491840.0,"bytes_per_snapshot":69836800.0,"last_snapshot_bytes":69836800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"} 2024-08-25T20:19:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:19:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:19:28' 2024-08-25T20:19:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:19:51.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:19:51.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:19:51.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3491840.0,"bytes_per_snapshot":69836800.0,"last_snapshot_bytes":69836800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"} 2024-08-25T20:19:51.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:19:51.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:51.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:19:51.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:51.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:19:51.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:19:51.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:19:51.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:19:51.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:51.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-08-25 20:19:28' 2024-08-25T20:19:51.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:51.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:51.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:19:51.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:51.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:19:28' 2024-08-25T20:19:51.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:51.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:51.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:19:51.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING441904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+replayingreplaying, {"bytes_per_second":3241369.6,"bytes_per_snapshot":64827392.0,"last_snapshot_bytes":64827392,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary7ce7df407-6dd2-4d59-8f98-8c52eaefd177up+replayingreplaying, {"bytes_per_second":3491840.0,"bytes_per_snapshot":69836800.0,"last_snapshot_bytes":69836800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+replayingreplaying, {"bytes_per_second":3236659.2,"bytes_per_snapshot":64733184.0,"last_snapshot_bytes":64733184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+replayingreplaying, {"bytes_per_second":3211673.6,"bytes_per_snapshot":64233472.0,"last_snapshot_bytes":64233472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28' 2024-08-25T20:19:51.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:19:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:19:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:51.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:19:51.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:19:51.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:51.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:19:51.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:51.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:51.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:51.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary7 2024-08-25T20:19:51.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:19:51.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:51.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:19:51.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-08-25T20:19:51.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-08-25T20:19:51.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:51.733+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3455040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:19:51.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:51.733+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973900 [3/ce7df407-6dd2-4d59-8f98-8c52eaefd177] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:19:53.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:19:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-08-25T20:19:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:19:53.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:19:53.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary7 2024-08-25T20:19:53.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:53.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary7 2024-08-25T20:19:53.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:19:53.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:19:53.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-08-25T20:19:53.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:53.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:53.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:19:53.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:19:53.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-08-25T20:19:53.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:53.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:53.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:19:53.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:53.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:53.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:53.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:53.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:19:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-08-25T20:19:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:19:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:19:54.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:19:54.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: cf4f7205-657c-4d10-848c-7bf5c56279da 2024-08-25T20:19:54.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:19:54.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3236659.2,"bytes_per_snapshot":64733184.0,"last_snapshot_bytes":64733184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"} 2024-08-25T20:19:54.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:19:54.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:19:28' 2024-08-25T20:19:54.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:19:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:19:54.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:19:54.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3236659.2,"bytes_per_snapshot":64733184.0,"last_snapshot_bytes":64733184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"} 2024-08-25T20:19:54.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:19:54.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:19:54.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:54.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:19:54.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:19:54.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:19:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:19:54.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:54.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-08-25 20:19:28' 2024-08-25T20:19:54.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:54.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:54.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:19:54.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:54.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:19:28' 2024-08-25T20:19:54.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:54.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:54.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:19:54.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING341904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+replayingreplaying, {"bytes_per_second":3241369.6,"bytes_per_snapshot":64827392.0,"last_snapshot_bytes":64827392,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary8cf4f7205-657c-4d10-848c-7bf5c56279daup+replayingreplaying, {"bytes_per_second":3236659.2,"bytes_per_snapshot":64733184.0,"last_snapshot_bytes":64733184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+replayingreplaying, {"bytes_per_second":3211673.6,"bytes_per_snapshot":64233472.0,"last_snapshot_bytes":64233472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28' 2024-08-25T20:19:54.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:19:54.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:19:54.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:54.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:19:54.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:19:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:19:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary8 2024-08-25T20:19:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:19:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:19:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-08-25T20:19:54.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-08-25T20:19:54.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:54.787+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4e6d860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:19:54.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:54.787+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973180 [3/cf4f7205-657c-4d10-848c-7bf5c56279da] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:19:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:19:56.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-08-25T20:19:56.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:56.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:56.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:19:56.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:56.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:19:56.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary8 2024-08-25T20:19:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary8 2024-08-25T20:19:56.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:19:56.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:19:56.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-08-25T20:19:56.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:56.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:56.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:19:56.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:19:56.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-08-25T20:19:56.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:56.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:56.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:19:56.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:56.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:56.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:56.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:56.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:19:57.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-08-25T20:19:57.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:19:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:19:57.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:19:57.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 975cc06f-b335-4181-8f15-5fd979f23a46 2024-08-25T20:19:57.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:19:57.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3211673.6,"bytes_per_snapshot":64233472.0,"last_snapshot_bytes":64233472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"} 2024-08-25T20:19:57.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:19:57.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:19:28' 2024-08-25T20:19:57.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:19:57.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:19:57.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:19:57.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3211673.6,"bytes_per_snapshot":64233472.0,"last_snapshot_bytes":64233472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"} 2024-08-25T20:19:57.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:19:57.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:57.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:19:57.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:57.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:19:57.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:19:57.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:19:57.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:19:57.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:57.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-08-25 20:19:28' 2024-08-25T20:19:57.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:57.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:57.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:19:57.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:57.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:19:28' 2024-08-25T20:19:57.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:19:57.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:19:57.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:19:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING241904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+replayingreplaying, {"bytes_per_second":3241369.6,"bytes_per_snapshot":64827392.0,"last_snapshot_bytes":64827392,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28image-primary9975cc06f-b335-4181-8f15-5fd979f23a46up+replayingreplaying, {"bytes_per_second":3211673.6,"bytes_per_snapshot":64233472.0,"last_snapshot_bytes":64233472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617115,"remote_snapshot_timestamp":1724617115,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:28' 2024-08-25T20:19:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:19:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:19:57.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:28' 2024-08-25T20:19:57.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:19:57.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:19:57.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:19:57.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:19:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:19:57.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:57.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:57.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary9 2024-08-25T20:19:57.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:19:57.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:57.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:19:57.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-08-25T20:19:57.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-08-25T20:19:57.918 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:57.918+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae36fe680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:19:57.919 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:57.918+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972500 [3/975cc06f-b335-4181-8f15-5fd979f23a46] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:19:58.330 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:58.329+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3378000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:19:58.330 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:19:58.329+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972500 [3/975cc06f-b335-4181-8f15-5fd979f23a46] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:19:59.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:19:59.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-08-25T20:19:59.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:59.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:59.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:19:59.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:59.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:19:59.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary9 2024-08-25T20:19:59.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:19:59.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary9 2024-08-25T20:19:59.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:19:59.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:19:59.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-08-25T20:19:59.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:59.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:59.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:19:59.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:19:59.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-08-25T20:19:59.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:19:59.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:19:59.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:19:59.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:19:59.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:19:59.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:19:59.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:19:59.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:20:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-08-25T20:20:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:20:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:20:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:20:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:20:00.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:20:00.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:20:00.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:20:00.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dda4aac1-b2d6-46d3-a987-574494886b5c 2024-08-25T20:20:00.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:20:00.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":64827392.0,"last_snapshot_bytes":64827392,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"} 2024-08-25T20:20:00.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:20:00.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:19:58' 2024-08-25T20:20:00.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:20:00.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:20:00.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:20:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":64827392.0,"last_snapshot_bytes":64827392,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"} 2024-08-25T20:20:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:20:00.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:20:00.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:20:00.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:20:00.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:20:00.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:20:00.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:20:00.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:20:00.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:58' 2024-08-25T20:20:00.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-08-25 20:19:58' 2024-08-25T20:20:00.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:00.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:00.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:00.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:20:00.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:19:58' 2024-08-25T20:20:00.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:20:00.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:20:00.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:20:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING141904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10dda4aac1-b2d6-46d3-a987-574494886b5cup+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":64827392.0,"last_snapshot_bytes":64827392,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617113,"remote_snapshot_timestamp":1724617113,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:19:58' 2024-08-25T20:20:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:20:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:20:00.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:19:58' 2024-08-25T20:20:00.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:20:00.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:20:00.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:20:00.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:20:00.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:20:00.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary10 2024-08-25T20:20:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:20:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-08-25T20:20:00.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-08-25T20:20:00.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:20:00.787+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade2d8d00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:20:00.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:20:00.787+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973680 [3/dda4aac1-b2d6-46d3-a987-574494886b5c] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:20:02.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:20:02.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-08-25T20:20:02.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:02.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:02.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:02.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:20:02.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:20:02.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary10 2024-08-25T20:20:02.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:02.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary10 2024-08-25T20:20:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..10} 2024-08-25T20:20:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS=() 2024-08-25T20:20:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS=() 2024-08-25T20:20:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS=() 2024-08-25T20:20:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS=() 2024-08-25T20:20:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RET=0 2024-08-25T20:20:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:20:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:20:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary1 2024-08-25T20:20:02.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-08-25T20:20:02.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:02.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:02.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:02.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:02.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:02.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:02.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:02.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:20:02.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:02.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-08-25T20:20:02.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:02.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:02.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:02.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:02.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:02.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:02.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:20:02.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:20:02.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:20:02.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:02.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-08-25T20:20:02.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:20:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:20:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-08-25T20:20:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-08-25T20:20:03.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:20:03.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-08-25T20:20:03.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary1': 2024-08-25T20:20:03.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:20:03.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:20:03.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:20:03.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 24c0132999c5 2024-08-25T20:20:03.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.24c0132999c5 2024-08-25T20:20:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:20:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:20:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:20:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:20:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:20:02 2024 2024-08-25T20:20:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:20:02 2024 2024-08-25T20:20:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:20:02 2024 2024-08-25T20:20:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:20:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:20:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 4d736788-0ad9-41ef-87c2-7b5bb9179356 2024-08-25T20:20:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:20:03.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:20:03.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:20:03.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-08-25T20:20:03.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd0 2024-08-25T20:20:03.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:20:03.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-08-25T20:20:03.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:20:03.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:20:03.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:20:03.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: c58ed92b-1a08-4971-b6fe-6337c89e7bdc 2024-08-25T20:20:03.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:20:03.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:20:03.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:03.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:20:03.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:20:03.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:20:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:20:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:03.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary1 2024-08-25T20:20:03.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-08-25T20:20:03.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:20:03.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary1 2024-08-25T20:20:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:20:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-08-25T20:20:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:20:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:20:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary1 2024-08-25T20:20:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:20:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary2 2024-08-25T20:20:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-08-25T20:20:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary1 2024-08-25T20:20:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:20:03.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:20:03.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:20:03.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:03.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-08-25T20:20:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:20:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:20:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-08-25T20:20:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:03.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:03.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:03.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-08-25T20:20:04.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:20:04.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-08-25T20:20:04.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1324 2024-08-25T20:20:04.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:20:04.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:04.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-08-25T20:20:04.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary2': 2024-08-25T20:20:04.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:20:04.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:20:04.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:20:04.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 24d0948240d3 2024-08-25T20:20:04.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.24d0948240d3 2024-08-25T20:20:04.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:20:04.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:20:04.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:20:04.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:20:04.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:20:03 2024 2024-08-25T20:20:04.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:20:03 2024 2024-08-25T20:20:04.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:20:03 2024 2024-08-25T20:20:04.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:20:04.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:20:04.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: dc6102c7-75ee-4e7a-a71e-9ba43c580e44 2024-08-25T20:20:04.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:20:04.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:20:04.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:20:04.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-08-25T20:20:04.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd1 2024-08-25T20:20:04.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:20:04.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-08-25T20:20:04.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:20:04.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:20:04.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:20:04.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 9e34949b-c50d-4a8b-91cd-b20c738ee08d 2024-08-25T20:20:04.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:20:04.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:20:04.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:04.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:20:04.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:20:04.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:20:04.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:20:04.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary2 2024-08-25T20:20:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-08-25T20:20:04.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:20:04.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary2 2024-08-25T20:20:04.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:20:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-08-25T20:20:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:20:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:20:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:20:04.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:20:04.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:20:04.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary3 2024-08-25T20:20:04.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-08-25T20:20:04.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary2 2024-08-25T20:20:04.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:04.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary2 2024-08-25T20:20:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:20:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:20:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:20:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-08-25T20:20:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-08-25T20:20:05.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:20:05.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:05.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:20:05.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:05.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-08-25T20:20:05.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:05.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:05.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:05.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:05.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-08-25T20:20:05.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:20:05.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-08-25T20:20:05.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:20:05.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1326 2024-08-25T20:20:05.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-08-25T20:20:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary3': 2024-08-25T20:20:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:20:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:20:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:20:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 24df6eb4de05 2024-08-25T20:20:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.24df6eb4de05 2024-08-25T20:20:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:20:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:20:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:20:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:20:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:20:05 2024 2024-08-25T20:20:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:20:05 2024 2024-08-25T20:20:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:20:05 2024 2024-08-25T20:20:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:20:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:20:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 7e83313d-862e-4e2d-bacb-d8647c2d63fb 2024-08-25T20:20:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:20:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:20:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:20:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-08-25T20:20:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd2 2024-08-25T20:20:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:20:05.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-08-25T20:20:05.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:20:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:20:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:20:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 6354a116-9bed-4bf9-acb5-e402fda4518e 2024-08-25T20:20:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:20:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:20:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:20:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:20:05.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:20:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:20:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:05.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary3 2024-08-25T20:20:05.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-08-25T20:20:05.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:20:05.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary3 2024-08-25T20:20:05.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:20:05.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-08-25T20:20:05.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary3 2024-08-25T20:20:05.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary4 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary3 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:20:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:20:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:20:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-08-25T20:20:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:20:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:20:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-08-25T20:20:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-08-25T20:20:06.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:20:06.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-08-25T20:20:06.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:20:06.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1328 2024-08-25T20:20:06.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:06.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-08-25T20:20:06.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary4': 2024-08-25T20:20:06.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:20:06.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:20:06.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:20:06.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 24eef8e1e57c 2024-08-25T20:20:06.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.24eef8e1e57c 2024-08-25T20:20:06.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:20:06.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:20:06.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:20:06.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:20:06.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:20:06 2024 2024-08-25T20:20:06.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:20:06 2024 2024-08-25T20:20:06.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:20:06 2024 2024-08-25T20:20:06.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:20:06.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:20:06.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: e620d71e-21fd-4188-a15b-b5bbf5db7712 2024-08-25T20:20:06.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:20:06.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:20:06.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:20:06.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-08-25T20:20:06.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd3 2024-08-25T20:20:06.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:20:06.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-08-25T20:20:06.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:20:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:20:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:20:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 78000558-5fd2-4ec3-b410-f50931b403a2 2024-08-25T20:20:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:20:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:20:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:20:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:20:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:20:06.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:20:06.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:06.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary4 2024-08-25T20:20:06.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-08-25T20:20:06.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:20:06.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary4 2024-08-25T20:20:06.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:20:06.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-08-25T20:20:06.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:06.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:06.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:06.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:06.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:20:06.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:06.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary5 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary4 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:20:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary4 2024-08-25T20:20:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-08-25T20:20:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-08-25T20:20:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:06.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:20:06.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:20:06.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:20:06.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:06.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-08-25T20:20:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:20:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:20:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-08-25T20:20:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-08-25T20:20:07.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:20:07.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-08-25T20:20:07.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:20:07.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1330 2024-08-25T20:20:07.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:07.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:07.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:07.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:07.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:07.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:07.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:07.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-08-25T20:20:07.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary5': 2024-08-25T20:20:07.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:20:07.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 24fda7bb7ffb 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.24fda7bb7ffb 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:20:06 2024 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:20:06 2024 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:20:06 2024 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: f5ba75f0-8101-4220-854b-b3b32ed57927 2024-08-25T20:20:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:20:07.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:20:07.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:20:07.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-08-25T20:20:07.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd4 2024-08-25T20:20:07.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:20:07.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-08-25T20:20:07.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:20:07.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:20:07.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:20:07.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 61e32183-172b-4ea6-823a-abeae4a81815 2024-08-25T20:20:07.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:20:07.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:20:07.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:07.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:20:07.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:20:07.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:20:07.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:20:07.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:07.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary5 2024-08-25T20:20:07.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-08-25T20:20:07.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:20:07.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary5 2024-08-25T20:20:07.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:20:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary5 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary6 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary5 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-08-25T20:20:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-08-25T20:20:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:20:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:20:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:20:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-08-25T20:20:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:20:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:20:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-08-25T20:20:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-08-25T20:20:08.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:20:08.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-08-25T20:20:08.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:20:08.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1332 2024-08-25T20:20:08.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1333 2024-08-25T20:20:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-08-25T20:20:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:20:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:20:08.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary6': 2024-08-25T20:20:08.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:20:08.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:20:08.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 250fe1a84932 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.250fe1a84932 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:20:07 2024 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:20:07 2024 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:20:07 2024 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 699752ba-8a9d-46f9-96ac-1842988c83f7 2024-08-25T20:20:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:20:08.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:20:08.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:20:08.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-08-25T20:20:08.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd5 2024-08-25T20:20:08.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:20:08.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-08-25T20:20:08.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:20:08.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:20:08.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:20:08.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 8de4cb6d-b2db-4086-a005-346a804f3b40 2024-08-25T20:20:08.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:20:08.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:20:08.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:08.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:20:08.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:20:08.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:20:08.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:20:08.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:08.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary6 2024-08-25T20:20:08.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-08-25T20:20:08.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:20:08.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary6 2024-08-25T20:20:08.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:20:08.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary7 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary6 2024-08-25T20:20:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary6 2024-08-25T20:20:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-08-25T20:20:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:20:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:20:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:20:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:08.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-08-25T20:20:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:20:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:20:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-08-25T20:20:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-08-25T20:20:09.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:20:09.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-08-25T20:20:09.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:09.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:20:09.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:09.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:09.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:09.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:20:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:20:09.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1335 2024-08-25T20:20:09.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1336 2024-08-25T20:20:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-08-25T20:20:09.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:09.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary7': 2024-08-25T20:20:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:20:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:20:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:20:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 252111e40a4b 2024-08-25T20:20:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.252111e40a4b 2024-08-25T20:20:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:20:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:20:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:20:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:20:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:20:08 2024 2024-08-25T20:20:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:20:08 2024 2024-08-25T20:20:09.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:20:08 2024 2024-08-25T20:20:09.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:20:09.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:20:09.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: e9b8f42f-7f86-45f5-b7cc-f7349a4ce8de 2024-08-25T20:20:09.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:20:09.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:20:09.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:20:09.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-08-25T20:20:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd6 2024-08-25T20:20:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:20:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-08-25T20:20:09.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:20:09.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:20:09.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:20:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 453b05c8-b21c-48fa-b6b5-86109a67236c 2024-08-25T20:20:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:20:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:20:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:20:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:20:09.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:20:09.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:20:09.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:09.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary7 2024-08-25T20:20:09.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-08-25T20:20:09.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:20:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary7 2024-08-25T20:20:09.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:20:09.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-08-25T20:20:09.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:20:09.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:20:09.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:20:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary8 2024-08-25T20:20:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-08-25T20:20:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary7 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:20:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:20:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:20:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-08-25T20:20:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:20:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary7 2024-08-25T20:20:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-08-25T20:20:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:20:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:20:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-08-25T20:20:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-08-25T20:20:10.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:20:10.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-08-25T20:20:10.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:10.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:20:10.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:10.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:10.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:10.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:20:10.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:20:10.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1338 2024-08-25T20:20:10.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1339 2024-08-25T20:20:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-08-25T20:20:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary8': 2024-08-25T20:20:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:20:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:20:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:20:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 253291452e6c 2024-08-25T20:20:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.253291452e6c 2024-08-25T20:20:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:20:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:20:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:20:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:20:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:20:09 2024 2024-08-25T20:20:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:20:09 2024 2024-08-25T20:20:10.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:20:09 2024 2024-08-25T20:20:10.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:20:10.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:20:10.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 87bb3060-f1c1-405d-a58d-d540f695a6ec 2024-08-25T20:20:10.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:20:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:20:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:20:10.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-08-25T20:20:10.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd7 2024-08-25T20:20:10.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:20:10.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-08-25T20:20:10.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:20:10.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:20:10.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:20:10.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 4f78a48e-b477-44bc-a04b-0ac0226fdc54 2024-08-25T20:20:10.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:20:10.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:20:10.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:10.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:20:10.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:20:10.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:20:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:20:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:10.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary8 2024-08-25T20:20:10.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-08-25T20:20:10.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:20:10.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary8 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:20:10.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary9 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary8 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary8 2024-08-25T20:20:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-08-25T20:20:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-08-25T20:20:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:20:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:20:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:20:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-08-25T20:20:10.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:20:10.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:10.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:20:10.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:10.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-08-25T20:20:10.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:10.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:10.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:10.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:10.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-08-25T20:20:11.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:20:11.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-08-25T20:20:11.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:20:11.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1341 2024-08-25T20:20:11.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1342 2024-08-25T20:20:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:11.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-08-25T20:20:11.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary9': 2024-08-25T20:20:11.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:20:11.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:20:11.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:20:11.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 25445710bb4 2024-08-25T20:20:11.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.25445710bb4 2024-08-25T20:20:11.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:20:11.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:20:11.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:20:11.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:20:11.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:20:10 2024 2024-08-25T20:20:11.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:20:10 2024 2024-08-25T20:20:11.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:20:10 2024 2024-08-25T20:20:11.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:20:11.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:20:11.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: be5f530f-f7f7-4c25-9be4-a1a854878e85 2024-08-25T20:20:11.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:20:11.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:20:11.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:20:11.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-08-25T20:20:11.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:11.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:20:11.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:11.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:11.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:11.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:20:11.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:11.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:11.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:11.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:11.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:11.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:11.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd8 2024-08-25T20:20:11.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:20:11.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-08-25T20:20:11.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:20:11.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:20:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:20:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 9d380b84-7480-4235-9a28-630efd0f342f 2024-08-25T20:20:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:20:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:20:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:20:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:20:11.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:20:11.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:20:11.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:11.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary9 2024-08-25T20:20:11.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-08-25T20:20:11.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:20:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary9 2024-08-25T20:20:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:20:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-08-25T20:20:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:20:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:20:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:20:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:20:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:20:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary10 2024-08-25T20:20:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-08-25T20:20:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary9 2024-08-25T20:20:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary9 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:20:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:20:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:20:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:20:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-08-25T20:20:11.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:20:11.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:11.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:20:11.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:20:11.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-08-25T20:20:11.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:11.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:11.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:11.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:20:11.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-08-25T20:20:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:20:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-08-25T20:20:12.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1344 2024-08-25T20:20:12.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:20:12.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1346 2024-08-25T20:20:12.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1345 2024-08-25T20:20:12.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:12.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:12.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-08-25T20:20:12.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:12.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:12.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:20:12.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:12.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:12.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:12.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:20:12.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:12.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:20:12.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:12.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:12.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:12.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:20:12.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary10': 2024-08-25T20:20:12.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:20:12.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:20:12.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:20:12.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 25599fd5a468 2024-08-25T20:20:12.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.25599fd5a468 2024-08-25T20:20:12.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:20:12.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:20:12.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:20:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:20:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:20:11 2024 2024-08-25T20:20:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:20:11 2024 2024-08-25T20:20:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:20:11 2024 2024-08-25T20:20:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:20:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:20:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 26a31609-06cb-4d50-b283-64d8473a46f9 2024-08-25T20:20:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:20:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:20:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:20:12.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-08-25T20:20:12.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd9 2024-08-25T20:20:12.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:20:12.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-08-25T20:20:12.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:20:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:20:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:20:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 0a7b1574-2ab7-4d9b-9f17-1699f2c4df64 2024-08-25T20:20:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:20:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:20:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:20:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:20:12.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:20:12.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:20:12.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:20:12.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary10 2024-08-25T20:20:12.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-08-25T20:20:12.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:20:12.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary10 2024-08-25T20:20:12.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:20:12.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-08-25T20:20:12.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:12.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:12.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:12.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:20:12.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:12.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:20:12.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:12.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:12.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:20:12.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:12.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 70954 2024-08-25T20:20:12.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:20:12.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary10 2024-08-25T20:20:12.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary10 2024-08-25T20:20:12.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-08-25T20:20:13.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:20:13.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-08-25T20:20:13.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1348 2024-08-25T20:20:13.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1350 2024-08-25T20:20:13.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1349 2024-08-25T20:20:13.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:13.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:13.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:13.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:13.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:20:13.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:13.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:13.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:13.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:20:13.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:13.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:20:13.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:13.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:13.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:13.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:20:14.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:20:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:20:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:20:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:20:14.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1352 2024-08-25T20:20:14.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1351 2024-08-25T20:20:14.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:14.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:15.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:15.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:15.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:15.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:15.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:15.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:15.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:20:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:20:15.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1353 2024-08-25T20:20:15.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:15.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:20:15.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:15.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:15.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:15.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:20:15.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1354 2024-08-25T20:20:15.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:15.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:16.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1356 2024-08-25T20:20:16.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1357 2024-08-25T20:20:16.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1355 2024-08-25T20:20:16.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:16.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:16.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:16.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:20:16.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:16.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:16.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:16.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:20:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:20:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:20:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:16.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:20:16.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:16.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:16.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:16.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:20:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1359 2024-08-25T20:20:17.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1358 2024-08-25T20:20:17.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1360 2024-08-25T20:20:17.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:17.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:17.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:17.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:17.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:20:17.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:17.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:17.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:17.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:20:17.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:17.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:20:17.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:17.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:17.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:17.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:20:18.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1362 2024-08-25T20:20:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:20:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:20:18.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1361 2024-08-25T20:20:18.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:18.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:20:18.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:18.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:18.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:18.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:20:18.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:18.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:19.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:19.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:19.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:19.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:19.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:19.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:19.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:19.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:20:19.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:19.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:19.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:19.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:20:19.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:19.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:20:19.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:19.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:19.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:19.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:20:19.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1363 2024-08-25T20:20:19.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1364 2024-08-25T20:20:19.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:19.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:20.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:20.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:20:20.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:20.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:20.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:20.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:20:20.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1365 2024-08-25T20:20:20.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:20.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:20:20.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:20.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:20.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:20.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:20:20.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:20.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:20:20.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:20.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:20.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:20.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:20:20.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1367 2024-08-25T20:20:20.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1366 2024-08-25T20:20:20.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:20.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:20.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1369 2024-08-25T20:20:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1370 2024-08-25T20:20:21.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:21.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:21.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1368 2024-08-25T20:20:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:21.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:21.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:20:21.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:21.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:21.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:21.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:20:21.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:21.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:20:21.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:21.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:21.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:21.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:20:22.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1371 2024-08-25T20:20:22.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:22.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1372 2024-08-25T20:20:22.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:22.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:20:22.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:22.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:22.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:22.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:20:22.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:20:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:20:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:23.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:23.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:20:23.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:23.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:23.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:23.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:20:23.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:23.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:20:23.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:23.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:23.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:23.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:20:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1373 2024-08-25T20:20:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:23.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1374 2024-08-25T20:20:23.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:24.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:24.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:20:24.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:24.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:24.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:24.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:20:24.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:24.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:20:24.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:24.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:24.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:24.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:20:24.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:24.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:20:24.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:24.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:24.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:24.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:20:24.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1375 2024-08-25T20:20:24.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:24.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1377 2024-08-25T20:20:24.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:25.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:25.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:20:25.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:25.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:25.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:25.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:20:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1379 2024-08-25T20:20:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:25.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1378 2024-08-25T20:20:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:25.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:25.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:20:25.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:25.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:25.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:25.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:20:26.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1376 2024-08-25T20:20:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:26.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1380 2024-08-25T20:20:26.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1381 2024-08-25T20:20:26.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:26.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:26.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:26.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:20:26.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:26.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:26.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:26.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:20:26.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:26.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:20:26.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:26.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:26.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:26.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:20:27.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1382 2024-08-25T20:20:27.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:27.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1384 2024-08-25T20:20:27.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:27.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:27.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:20:27.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:27.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:27.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:27.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:20:27.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:27.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:20:27.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:27.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:27.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:27.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:20:28.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1383 2024-08-25T20:20:28.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:28.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1385 2024-08-25T20:20:28.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:28.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:28.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:20:28.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:28.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:28.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:28.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:20:28.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:28.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:20:28.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:28.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:28.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:28.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:20:29.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:29.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:29.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:29.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:29.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:29.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:29.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:29.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:20:29.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:29.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:29.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:29.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:20:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:20:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:20:29.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1387 2024-08-25T20:20:29.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:30.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1386 2024-08-25T20:20:30.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:30.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:30.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:20:30.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:30.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:30.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:30.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:20:30.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:30.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:20:30.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:30.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:30.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:30.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:20:30.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1389 2024-08-25T20:20:30.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:31.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1388 2024-08-25T20:20:31.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:20:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:20:31.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:31.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:20:31.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:31.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:31.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:31.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:20:32.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1391 2024-08-25T20:20:32.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1390 2024-08-25T20:20:32.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:32.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:32.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1393 2024-08-25T20:20:32.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1392 2024-08-25T20:20:32.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:32.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:20:32.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:32.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:32.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:32.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:20:33.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:33.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:20:33.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:33.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:33.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:33.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:20:33.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1395 2024-08-25T20:20:33.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1394 2024-08-25T20:20:33.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:33.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:33.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:33.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:33.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:33.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:33.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:33.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:34.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1396 2024-08-25T20:20:34.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:34.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:20:34.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:34.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:34.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:34.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:20:34.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:34.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1397 2024-08-25T20:20:34.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:20:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:35.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:35.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:35.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:20:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1399 2024-08-25T20:20:35.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:35.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:20:35.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:35.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:35.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:35.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:20:35.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:35.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:35.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:20:35.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:35.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:35.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:35.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:20:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:20:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:20:36.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1398 2024-08-25T20:20:36.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:36.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1401 2024-08-25T20:20:36.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1400 2024-08-25T20:20:36.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:36.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:20:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:20:36.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:36.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:20:36.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:36.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:36.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:36.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:20:37.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:37.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:20:37.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:37.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:37.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:37.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:20:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1402 2024-08-25T20:20:37.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:37.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:20:37.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:37.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:37.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:37.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:20:37.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:38.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1403 2024-08-25T20:20:38.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:38.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:20:38.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:38.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:38.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:38.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:20:38.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:39.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:39.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1405 2024-08-25T20:20:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1404 2024-08-25T20:20:39.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:39.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:20:39.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:39.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:39.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:39.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:20:39.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:39.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:39.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:20:39.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:39.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:39.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:39.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:20:39.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:40.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1406 2024-08-25T20:20:40.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1407 2024-08-25T20:20:40.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:40.451 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:20:40.450+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae504c800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:20:40.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:40.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:40.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:20:40.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:40.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:40.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:40.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:20:40.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1409 2024-08-25T20:20:40.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:41.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1408 2024-08-25T20:20:41.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1411 2024-08-25T20:20:41.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:41.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1410 2024-08-25T20:20:41.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:41.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:20:41.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:41.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:41.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:41.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:20:41.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:41.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:20:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:42.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:20:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:20:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:20:42.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1413 2024-08-25T20:20:42.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:43.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1412 2024-08-25T20:20:43.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:43.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:43.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:20:43.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:43.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:43.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:43.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:20:43.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1415 2024-08-25T20:20:43.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1414 2024-08-25T20:20:43.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:43.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:20:43.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:43.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:43.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:43.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:20:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:43.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:43.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:43.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:43.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:43.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:43.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:43.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:44.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:44.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:20:44.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:44.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:44.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:44.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:20:44.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1417 2024-08-25T20:20:44.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1416 2024-08-25T20:20:44.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:20:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:20:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:20:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:20:45.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1419 2024-08-25T20:20:45.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:20:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:20:46.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1418 2024-08-25T20:20:46.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:46.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:46.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:20:46.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:46.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:46.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:46.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:20:46.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:46.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:20:46.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:46.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:46.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:46.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:20:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:20:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:20:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1421 2024-08-25T20:20:47.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1420 2024-08-25T20:20:47.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:47.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:47.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:20:47.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:47.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:47.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:47.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:20:47.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:47.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:20:47.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:47.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:47.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:47.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:20:47.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:47.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1423 2024-08-25T20:20:47.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:48.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1422 2024-08-25T20:20:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:20:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:20:49.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:49.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:49.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:49.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:49.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:49.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:49.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1425 2024-08-25T20:20:49.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1424 2024-08-25T20:20:49.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:49.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1428 2024-08-25T20:20:49.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:50.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1426 2024-08-25T20:20:50.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1427 2024-08-25T20:20:50.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:50.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:20:50.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:50.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:50.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:50.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:20:50.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:50.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:20:50.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:50.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:50.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:50.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:20:50.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:50.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:20:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:20:50.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1429 2024-08-25T20:20:50.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:51.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1432 2024-08-25T20:20:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1430 2024-08-25T20:20:51.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:51.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:20:51.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:51.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:51.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:51.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:20:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:20:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:20:52.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1433 2024-08-25T20:20:52.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:20:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:20:52.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:52.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:20:52.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:52.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:52.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:52.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:20:53.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1431 2024-08-25T20:20:53.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:53.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:53.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:20:53.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:53.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:53.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:53.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:20:53.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:53.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:20:53.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:53.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:53.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:53.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:20:53.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1434 2024-08-25T20:20:53.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1435 2024-08-25T20:20:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1436 2024-08-25T20:20:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:53.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:53.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:53.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:53.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:53.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:53.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:54.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:54.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:20:54.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:54.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:54.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:20:54.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:20:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:54.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:20:54.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:54.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:54.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:20:54.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:20:54.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1438 2024-08-25T20:20:54.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1437 2024-08-25T20:20:54.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:54.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:55.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:55.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:20:55.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:55.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:55.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:20:55.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:20:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1439 2024-08-25T20:20:56.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:56.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:56.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:20:56.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:56.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:56.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:20:56.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:20:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:20:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:20:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:20:56.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:56.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:20:56.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:56.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:56.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:20:56.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:20:56.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:56.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:20:56.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:56.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:56.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:20:56.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:20:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1441 2024-08-25T20:20:57.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1440 2024-08-25T20:20:57.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:57.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:57.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1443 2024-08-25T20:20:57.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1445 2024-08-25T20:20:57.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:57.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:20:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:20:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:20:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:20:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:20:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:20:58.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1442 2024-08-25T20:20:58.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:20:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:20:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:20:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:20:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:20:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:20:59.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1444 2024-08-25T20:20:59.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1446 2024-08-25T20:20:59.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:20:59.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:00.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:00.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:21:00.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:00.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:00.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:21:00.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:21:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:21:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:21:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:21:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:00.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1447 2024-08-25T20:21:00.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:00.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:21:00.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:00.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:00.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:21:00.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:21:00.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1448 2024-08-25T20:21:00.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:00.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:00.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1449 2024-08-25T20:21:00.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:01.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1451 2024-08-25T20:21:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1450 2024-08-25T20:21:01.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:21:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:21:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:21:01.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:02.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1452 2024-08-25T20:21:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1454 2024-08-25T20:21:02.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:02.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:02.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:02.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:02.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:02.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:02.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:03.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1453 2024-08-25T20:21:03.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:03.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1455 2024-08-25T20:21:03.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1456 2024-08-25T20:21:03.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:03.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:03.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:03.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:21:03.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:03.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:03.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:21:03.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:21:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:21:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:21:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:21:03.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:03.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:21:03.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:03.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:03.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:21:03.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:21:04.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1457 2024-08-25T20:21:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1458 2024-08-25T20:21:04.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:04.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:21:04.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:04.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:04.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:21:04.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:21:04.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:04.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:21:04.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:04.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:04.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:21:04.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:21:04.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:04.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:05.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:05.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:05.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:05.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:05.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:05.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:05.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:05.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:21:05.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:05.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:05.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:21:05.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:21:06.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1459 2024-08-25T20:21:06.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:06.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:21:06.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:06.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:06.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:21:06.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:21:06.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:06.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:06.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:06.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:06.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:06.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:06.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:07.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1460 2024-08-25T20:21:07.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1461 2024-08-25T20:21:07.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:07.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:21:07.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:07.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:07.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:21:07.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:21:07.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:07.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:07.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:21:07.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:07.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:07.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:21:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:21:07.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1462 2024-08-25T20:21:07.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:07.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1463 2024-08-25T20:21:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:09.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1465 2024-08-25T20:21:09.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1466 2024-08-25T20:21:09.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:09.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:21:09.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:09.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:09.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:21:09.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:21:09.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:09.461 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:21:09.460+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf88a800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:21:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:10.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1464 2024-08-25T20:21:10.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1468 2024-08-25T20:21:10.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1467 2024-08-25T20:21:10.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:10.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:21:10.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:10.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:10.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:21:10.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:21:10.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:10.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:10.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:21:10.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:10.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:10.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:21:10.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:21:10.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1469 2024-08-25T20:21:10.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:11.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1471 2024-08-25T20:21:11.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1470 2024-08-25T20:21:11.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:11.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:11.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:11.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:21:11.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:11.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:11.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:21:11.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:21:12.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1473 2024-08-25T20:21:12.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:12.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:12.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:12.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:12.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:12.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:12.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:12.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:12.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:12.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:12.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1472 2024-08-25T20:21:13.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:13.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1474 2024-08-25T20:21:13.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:13.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:21:13.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:13.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:13.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:21:13.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:21:13.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:13.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:21:13.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:13.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:13.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:21:13.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:21:13.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1475 2024-08-25T20:21:13.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:13.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:21:13.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:13.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:13.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:21:13.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:21:13.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:13.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:13.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:13.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:21:13.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:13.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:13.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:21:13.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:21:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:21:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:21:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:21:14.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:14.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:21:14.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:14.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:14.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:21:14.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:21:14.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1477 2024-08-25T20:21:14.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1476 2024-08-25T20:21:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:14.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:21:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:15.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:21:15.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:21:16.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1478 2024-08-25T20:21:16.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1479 2024-08-25T20:21:16.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:16.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:16.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:16.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:16.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:16.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:16.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:16.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:16.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:16.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:16.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:16.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:16.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:16.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:16.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:16.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:16.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:16.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:16.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:16.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:17.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1480 2024-08-25T20:21:17.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1481 2024-08-25T20:21:17.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:17.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:21:17.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:17.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:17.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:21:17.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:21:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:21:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:21:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:21:17.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:17.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:17.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1483 2024-08-25T20:21:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:18.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1482 2024-08-25T20:21:18.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:18.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1487 2024-08-25T20:21:18.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:19.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:19.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:21:19.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:19.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:19.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:21:19.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:21:19.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1485 2024-08-25T20:21:19.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:19.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:21:19.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:19.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:19.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:21:19.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:21:19.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1484 2024-08-25T20:21:19.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:19.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:20.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1486 2024-08-25T20:21:20.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:20.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1489 2024-08-25T20:21:20.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1488 2024-08-25T20:21:20.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:20.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:21:20.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:20.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:20.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:21:20.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:21:20.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:20.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:20.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:20.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:21:20.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:20.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:20.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:21:20.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:21:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:21.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1490 2024-08-25T20:21:21.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1491 2024-08-25T20:21:21.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:21.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:21.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:21.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:21:21.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:21.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:21.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:21:21.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:21:21.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:21:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:21:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:21:22.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1493 2024-08-25T20:21:22.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1494 2024-08-25T20:21:22.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:22.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:22.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:22.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:22.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:22.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:22.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:22.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:23.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1492 2024-08-25T20:21:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:23.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1496 2024-08-25T20:21:23.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1495 2024-08-25T20:21:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:23.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:23.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:23.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:21:23.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:23.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:23.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:21:23.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:21:23.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:23.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:21:23.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:23.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:23.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:21:23.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:21:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:21:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:21:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:21:24.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1497 2024-08-25T20:21:24.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:21:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:21:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:21:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:21:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:21:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:21:25.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:25.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:21:25.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:25.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:25.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:21:25.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:21:25.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:25.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:21:25.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:25.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:25.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:21:25.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:21:26.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1499 2024-08-25T20:21:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:26.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1498 2024-08-25T20:21:26.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:26.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:26.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:26.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:26.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:26.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:26.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:26.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:26.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:26.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:26.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:26.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:26.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:26.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:26.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1503 2024-08-25T20:21:26.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:27.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1501 2024-08-25T20:21:27.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1500 2024-08-25T20:21:27.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:27.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:21:27.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:27.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:27.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:21:27.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:21:27.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:27.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:27.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1504 2024-08-25T20:21:27.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:28.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1502 2024-08-25T20:21:28.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1506 2024-08-25T20:21:28.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:28.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:28.441 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:21:28.439+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf3abc00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:21:29.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1505 2024-08-25T20:21:29.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:29.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:21:29.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:29.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:29.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:21:29.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:21:29.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:29.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:29.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:21:29.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:29.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:29.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:21:29.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:21:29.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:29.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:21:29.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:29.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:29.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:21:29.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:21:30.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1507 2024-08-25T20:21:30.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:21:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:21:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:21:30.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1509 2024-08-25T20:21:30.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:30.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:21:30.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:30.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:30.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:21:30.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:21:30.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1508 2024-08-25T20:21:30.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:30.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:30.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:30.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:30.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:30.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:30.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:30.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:31.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1510 2024-08-25T20:21:31.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:31.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1512 2024-08-25T20:21:31.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1511 2024-08-25T20:21:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:31.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:31.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:31.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:21:31.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:31.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:31.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:21:31.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:21:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:32.408 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:21:32.406+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf917c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:21:32.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:32.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:32.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:32.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:32.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:32.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:32.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1514 2024-08-25T20:21:32.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1513 2024-08-25T20:21:33.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:33.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:21:33.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:33.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:33.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:21:33.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:21:33.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:33.426 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:21:33.425+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adfa69400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:21:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1516 2024-08-25T20:21:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:21:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:21:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:21:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:21:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:21:33.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:21:33.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:34.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1515 2024-08-25T20:21:34.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:34.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:21:34.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:34.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:34.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:21:34.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:21:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:21:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:21:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:21:34.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:34.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:21:34.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:34.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:34.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:21:34.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:21:34.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:34.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1517 2024-08-25T20:21:34.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:35.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:35.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:21:35.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:35.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:35.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:21:35.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:21:36.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1519 2024-08-25T20:21:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1518 2024-08-25T20:21:36.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:36.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:36.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:36.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:36.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:36.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:36.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:36.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:37.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1521 2024-08-25T20:21:37.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1522 2024-08-25T20:21:37.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:37.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:37.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:37.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:37.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:37.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:37.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:37.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:37.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:37.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:21:37.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:37.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:37.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:21:37.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:21:37.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1525 2024-08-25T20:21:37.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1524 2024-08-25T20:21:37.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:37.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:38.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1523 2024-08-25T20:21:38.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:38.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1526 2024-08-25T20:21:38.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:39.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:21:39.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:39.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:39.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:21:39.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:21:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:21:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:21:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:21:39.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1520 2024-08-25T20:21:39.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:40.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1527 2024-08-25T20:21:40.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:40.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:40.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:21:40.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:40.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:40.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:21:40.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:21:40.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:40.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:21:40.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:40.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:40.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:21:40.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:21:40.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1529 2024-08-25T20:21:40.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:40.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1528 2024-08-25T20:21:40.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:40.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:41.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1530 2024-08-25T20:21:41.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1532 2024-08-25T20:21:41.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1531 2024-08-25T20:21:41.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:41.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1533 2024-08-25T20:21:41.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:41.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:41.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:41.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:41.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:21:41.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:41.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:41.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:21:41.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:21:41.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:41.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:41.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:41.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:41.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:41.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:42.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1535 2024-08-25T20:21:42.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1534 2024-08-25T20:21:42.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:42.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:21:42.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:42.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:42.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:21:42.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:21:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:43.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:43.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:21:43.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:43.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:43.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:21:43.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:21:43.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:43.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:21:43.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:43.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:43.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:21:43.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:21:43.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:43.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:21:43.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:43.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:43.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:21:43.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:21:44.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:44.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:44.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:44.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:44.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:44.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:44.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:44.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:21:44.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:44.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:44.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:21:44.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:21:44.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:44.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:44.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:44.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:44.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:44.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:44.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:44.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:21:44.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:44.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:44.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:21:44.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:21:44.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1537 2024-08-25T20:21:44.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:45.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1536 2024-08-25T20:21:45.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:45.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:45.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:45.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:45.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:45.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:45.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:45.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:21:45.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:45.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:45.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:21:45.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:21:45.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:45.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1541 2024-08-25T20:21:45.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1540 2024-08-25T20:21:45.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:45.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:46.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1539 2024-08-25T20:21:46.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1538 2024-08-25T20:21:46.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:46.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:46.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1545 2024-08-25T20:21:46.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:47.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1542 2024-08-25T20:21:47.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1543 2024-08-25T20:21:47.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:47.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:21:47.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:47.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:47.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:21:47.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:21:47.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:47.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:48.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1544 2024-08-25T20:21:48.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:21:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:21:48.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:21:48.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:48.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:48.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:48.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:48.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:48.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:48.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:49.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:49.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:21:49.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:49.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:49.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:21:49.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:21:49.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:49.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:21:49.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:49.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:49.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:21:49.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:21:49.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:49.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:49.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:49.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:49.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:49.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:49.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1547 2024-08-25T20:21:49.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1549 2024-08-25T20:21:49.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:49.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:50.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1548 2024-08-25T20:21:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1546 2024-08-25T20:21:50.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:21:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:21:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:21:50.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:50.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:21:50.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:50.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:50.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:21:50.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:21:50.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1552 2024-08-25T20:21:50.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1550 2024-08-25T20:21:50.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1551 2024-08-25T20:21:50.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:50.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:50.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:51.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1554 2024-08-25T20:21:51.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1553 2024-08-25T20:21:51.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:51.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:51.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:51.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:21:51.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:51.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:51.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:21:51.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:21:52.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1555 2024-08-25T20:21:52.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:21:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:21:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:21:53.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:53.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:53.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:53.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:53.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:53.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:53.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:53.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:21:53.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:53.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:53.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:53.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:53.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:53.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:53.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:53.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:21:53.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:53.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:53.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:21:53.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:21:53.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:53.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:21:53.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:53.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:53.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:21:53.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:21:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:21:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:21:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:21:54.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:54.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:21:54.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:21:54.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:21:54.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1557 2024-08-25T20:21:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1558 2024-08-25T20:21:54.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:55.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1559 2024-08-25T20:21:55.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1560 2024-08-25T20:21:55.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1556 2024-08-25T20:21:55.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:55.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:21:55.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:55.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:55.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:21:55.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:21:55.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:55.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:55.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:56.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1561 2024-08-25T20:21:56.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1562 2024-08-25T20:21:56.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:56.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:57.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1563 2024-08-25T20:21:57.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1564 2024-08-25T20:21:57.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:57.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:57.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:57.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:21:57.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:57.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:57.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:21:57.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:21:57.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:57.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:21:57.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:57.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:57.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:21:57.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:21:58.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1565 2024-08-25T20:21:58.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:21:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:21:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:21:58.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:58.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:21:58.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:58.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:58.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:21:58.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:21:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:21:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:21:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:21:59.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1566 2024-08-25T20:21:59.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:21:59.388+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf98e000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:21:59.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:21:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:59.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:21:59.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:59.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:59.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:21:59.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:21:59.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:21:59.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:21:59.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:21:59.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:21:59.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:21:59.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:21:59.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1570 2024-08-25T20:21:59.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1569 2024-08-25T20:22:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1567 2024-08-25T20:22:00.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1568 2024-08-25T20:22:00.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:00.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:22:00.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:00.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:00.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:22:00.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:22:00.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:00.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:22:00.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:00.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:00.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:22:00.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:22:00.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:00.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:00.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1571 2024-08-25T20:22:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1572 2024-08-25T20:22:00.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:00.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:01.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1573 2024-08-25T20:22:01.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:01.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:22:01.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:01.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:01.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:22:01.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:22:01.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1574 2024-08-25T20:22:01.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:01.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:02.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1575 2024-08-25T20:22:02.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:22:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:22:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:22:02.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:02.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:22:02.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:02.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:02.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:22:02.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:22:03.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:03.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:22:03.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:03.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:03.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:22:03.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:22:03.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:03.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:22:03.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:03.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:03.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:22:03.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:22:03.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:03.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:22:03.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:03.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:03.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:22:03.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:22:03.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:22:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:22:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:22:03.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:03.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:22:03.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:03.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:03.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:22:03.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:22:04.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:04.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:22:04.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:04.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:04.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:22:04.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:22:04.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:04.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:22:04.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:04.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:04.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:22:04.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:22:04.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1580 2024-08-25T20:22:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1577 2024-08-25T20:22:05.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:05.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:22:05.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:05.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:22:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:22:05.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:06.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1582 2024-08-25T20:22:06.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1581 2024-08-25T20:22:06.438 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:22:06.437+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf8b1000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:22:06.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:06.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:07.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1584 2024-08-25T20:22:07.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1576 2024-08-25T20:22:07.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1583 2024-08-25T20:22:07.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:07.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:07.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:07.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:22:07.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:07.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:07.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:22:07.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:22:08.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1585 2024-08-25T20:22:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:22:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:08.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:08.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:22:08.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:22:08.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1586 2024-08-25T20:22:08.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:09.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1578 2024-08-25T20:22:09.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1579 2024-08-25T20:22:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:22:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:22:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:22:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:09.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:22:09.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:09.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:09.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:22:09.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:22:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:09.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1587 2024-08-25T20:22:09.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:22:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:22:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:22:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1589 2024-08-25T20:22:10.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1588 2024-08-25T20:22:10.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:10.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:10.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:22:10.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:10.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:10.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:22:10.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:22:10.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:10.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:22:10.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:10.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:10.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:22:10.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:22:10.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:11.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1592 2024-08-25T20:22:11.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1591 2024-08-25T20:22:11.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:11.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:11.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:11.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:22:11.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:11.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:11.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:22:11.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:22:11.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:11.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:22:11.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:11.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:11.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:22:11.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:22:12.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1593 2024-08-25T20:22:12.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1594 2024-08-25T20:22:12.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:12.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:12.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:12.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:22:12.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:12.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:12.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:22:12.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:22:12.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:12.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:22:12.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:12.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:12.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:22:12.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:22:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:22:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:22:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:22:13.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1590 2024-08-25T20:22:13.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:13.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:13.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:22:13.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:13.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:13.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:22:13.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:22:13.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1596 2024-08-25T20:22:13.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:13.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:22:13.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:13.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:13.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:22:13.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:22:13.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1595 2024-08-25T20:22:13.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:13.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:22:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:22:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:22:14.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:14.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:22:14.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:14.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:22:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:22:14.458 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:22:14.455+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae3a86400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:22:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:22:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:22:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:22:15.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:15.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:22:15.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:15.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:15.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:22:15.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:22:15.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1597 2024-08-25T20:22:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:15.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1600 2024-08-25T20:22:15.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:16.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1599 2024-08-25T20:22:16.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1598 2024-08-25T20:22:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:16.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:16.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1603 2024-08-25T20:22:16.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:16.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:22:16.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:16.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:16.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:22:16.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:22:16.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:16.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:16.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:22:16.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:16.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:16.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:22:16.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:22:16.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:22:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:22:16.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:22:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1601 2024-08-25T20:22:17.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:17.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1604 2024-08-25T20:22:17.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1602 2024-08-25T20:22:18.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:18.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:18.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:22:18.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:18.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:18.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:22:18.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:22:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:22:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:22:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:22:19.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:19.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:22:19.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:19.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:19.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:22:19.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:22:19.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:19.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:22:19.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:19.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:19.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:22:19.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:22:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1606 2024-08-25T20:22:19.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:19.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:22:19.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:19.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:19.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:22:19.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:22:19.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1605 2024-08-25T20:22:19.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:19.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1607 2024-08-25T20:22:19.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1608 2024-08-25T20:22:19.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:19.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1609 2024-08-25T20:22:20.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:20.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:22:20.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:20.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:20.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:22:20.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:22:20.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1610 2024-08-25T20:22:20.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:20.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:20.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:20.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:22:20.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:20.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:20.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:22:20.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:22:21.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1611 2024-08-25T20:22:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:21.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1612 2024-08-25T20:22:21.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:21.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:21.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:22:21.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:21.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:21.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:22:21.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:22:22.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1613 2024-08-25T20:22:22.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:22.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1614 2024-08-25T20:22:22.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:22.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:22.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:22:22.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:22.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:22.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:22:22.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:22:22.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:22.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:22:22.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:22.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:22.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:22:22.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:22:22.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:22.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:22:22.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:22.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:22.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:22:22.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:22:22.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:22.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:22:22.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:22.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:22.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:22:22.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:22:23.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1617 2024-08-25T20:22:23.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1618 2024-08-25T20:22:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1615 2024-08-25T20:22:23.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1616 2024-08-25T20:22:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:23.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:23.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:22:23.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:23.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:23.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:22:23.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:22:23.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:23.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:23.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:22:23.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:22:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:22:23.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:24.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:24.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:22:24.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:24.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:24.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:22:24.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:22:24.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:24.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:22:24.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:24.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:24.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:22:24.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:22:25.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:22:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:22:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:22:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:22:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:22:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:22:26.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1619 2024-08-25T20:22:26.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1621 2024-08-25T20:22:26.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1620 2024-08-25T20:22:26.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:26.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:26.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:26.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:22:26.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:26.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:26.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:22:26.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:22:26.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:26.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:22:26.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:26.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:26.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:22:26.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:22:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:22:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:22:26.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:22:27.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1622 2024-08-25T20:22:27.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:27.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1625 2024-08-25T20:22:27.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:28.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1623 2024-08-25T20:22:28.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1624 2024-08-25T20:22:28.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:28.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:29.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1626 2024-08-25T20:22:29.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1627 2024-08-25T20:22:29.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:29.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:29.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:22:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:22:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:22:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:22:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71086 2024-08-25T20:22:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:22:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71218 2024-08-25T20:22:30.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1628 2024-08-25T20:22:30.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:22:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:22:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:22:30.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:30.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:22:30.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:30.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:30.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:22:30.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:22:31.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1629 2024-08-25T20:22:31.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:22:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:22:31.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:22:31.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:22:31.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71350 2024-08-25T20:22:32.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:22:32.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:22:32.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:22:32.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:22:32.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:22:32.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:22:32.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:22:32.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71502 2024-08-25T20:22:32.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:22:32.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71655 2024-08-25T20:22:32.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1631 2024-08-25T20:22:32.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:33.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1630 2024-08-25T20:22:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:33.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:22:33.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71808 2024-08-25T20:22:33.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1632 2024-08-25T20:22:33.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:22:34.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:22:34.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71961 2024-08-25T20:22:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:22:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 72134 2024-08-25T20:22:36.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:22:36.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 72308 2024-08-25T20:22:36.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:22:36.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:22:36.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 70955 2024-08-25T20:25:04.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:25:04.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:25:04.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:25:04.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71087 2024-08-25T20:25:05.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:25:05.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:25:05.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:25:05.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71219 2024-08-25T20:25:06.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:25:06.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:25:06.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:25:06.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71351 2024-08-25T20:25:07.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:25:07.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:25:07.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:25:07.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71503 2024-08-25T20:25:08.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:25:08.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:25:08.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:25:08.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71656 2024-08-25T20:25:09.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:25:09.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:25:09.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:25:09.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71809 2024-08-25T20:25:10.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:25:10.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:25:10.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:25:10.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 71962 2024-08-25T20:25:11.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:25:11.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:25:11.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:25:11.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 72135 2024-08-25T20:25:12.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:25:12.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:25:12.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:25:12.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 72309 2024-08-25T20:25:13.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:25:13.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:25:13.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:25:13.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:25:13.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:25:13.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:25:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 1 2024-08-25T20:25:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd0 2024-08-25T20:25:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary1 2024-08-25T20:25:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary1 2024-08-25T20:25:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:25:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary1 2024-08-25T20:25:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:25:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:25:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 2 2024-08-25T20:25:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd1 2024-08-25T20:25:13.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary2 2024-08-25T20:25:13.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary2 2024-08-25T20:25:13.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:25:13.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary2 2024-08-25T20:25:13.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:25:13.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:25:13.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 3 2024-08-25T20:25:13.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd2 2024-08-25T20:25:13.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary3 2024-08-25T20:25:13.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary3 2024-08-25T20:25:13.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:25:13.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary3 2024-08-25T20:25:13.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:25:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:25:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 4 2024-08-25T20:25:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd3 2024-08-25T20:25:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary4 2024-08-25T20:25:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary4 2024-08-25T20:25:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:25:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:25:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary4 2024-08-25T20:25:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:25:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 5 2024-08-25T20:25:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd4 2024-08-25T20:25:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary5 2024-08-25T20:25:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary5 2024-08-25T20:25:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:25:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary5 2024-08-25T20:25:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:25:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:25:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 6 2024-08-25T20:25:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd5 2024-08-25T20:25:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary6 2024-08-25T20:25:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary6 2024-08-25T20:25:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:25:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary6 2024-08-25T20:25:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:25:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 7 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd6 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary7 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary7 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary7 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 8 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd7 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary8 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary8 2024-08-25T20:25:13.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:25:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary8 2024-08-25T20:25:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:25:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:25:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:25:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 78268 2024-08-25T20:25:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 9 2024-08-25T20:25:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd8 2024-08-25T20:25:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary9 2024-08-25T20:25:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary9 2024-08-25T20:25:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:25:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary9 2024-08-25T20:25:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 10 2024-08-25T20:25:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd9 2024-08-25T20:25:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary10 2024-08-25T20:25:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary10 2024-08-25T20:25:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:25:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary10 2024-08-25T20:25:13.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:25:13.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:25:13.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:25:13.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:25:14.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:25:14.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:25:14.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:25:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:25:14.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:25:14.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:25:14.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:25:14.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:25:14.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:25:14.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:25:14.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:25:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:25:14.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:25:14.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:25:14.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:25:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:26:04.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=404de2bf7083c6cbb81a334f9a25661b 2024-08-25T20:26:04.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-08-25T20:26:05.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary5 2024-08-25T20:26:05.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:05.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:05.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:26:05.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-08-25T20:26:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=e31838f8dca8c47c076c06c9b2192cc8 2024-08-25T20:26:05.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-08-25T20:26:05.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary9 2024-08-25T20:26:05.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:05.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:05.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:26:05.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-08-25T20:26:06.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=cdd2aad006dc5e72b0a9921228e29eb4 2024-08-25T20:26:06.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-08-25T20:26:06.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary10 2024-08-25T20:26:06.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:06.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:06.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:26:06.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-08-25T20:26:06.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=e267dfc07734ad1fea743b1ab55d1f60 2024-08-25T20:26:06.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-08-25T20:26:07.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary6 2024-08-25T20:26:07.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:26:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-08-25T20:26:07.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:26:07.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-08-25T20:26:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:26:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:08.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:26:08.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-08-25T20:26:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:26:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:08.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:08.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:08.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:08.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:08.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:26:08.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:08.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:08.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:26:08.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:08.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:08.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:08.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:26:09.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=3afd7f865d4a38ef3c819b723fddc2e2 2024-08-25T20:26:09.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-08-25T20:26:09.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:26:09.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: be5f530f-f7f7-4c25-9be4-a1a854878e85 2024-08-25T20:26:09.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:09.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:09.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:09.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:09.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:09.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:09.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:09.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:26:09.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary7 2024-08-25T20:26:09.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:09.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:09.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:26:09.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-08-25T20:26:09.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=34a43060c745963fa84ea70b378a5119 2024-08-25T20:26:09.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-08-25T20:26:09.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary3 2024-08-25T20:26:09.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:09.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:09.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:26:09.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-08-25T20:26:09.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=1cde6aaa698efd5a71cfe6bc2dba2ede 2024-08-25T20:26:09.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-08-25T20:26:09.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:26:09.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:26:09.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-08-25T20:26:09.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:09.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:09.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:26:09.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:09.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:09.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:09.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:09.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:09.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-08-25T20:26:09.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:09.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:09.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:26:09.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:09.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:09.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:09.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:09.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:09.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=0032916ffb088c625320414bc8abd842 2024-08-25T20:26:09.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-08-25T20:26:09.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary4 2024-08-25T20:26:09.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:09.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:09.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:26:09.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-08-25T20:26:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:26:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:26:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:09.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:09.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:26:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary1 2024-08-25T20:26:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:26:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-08-25T20:26:09.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:26:09.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 699752ba-8a9d-46f9-96ac-1842988c83f7 2024-08-25T20:26:09.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:09.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:09.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:09.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:09.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:09.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:09.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:09.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:26:10.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=2775cbd0a44ee877a50b5d89fa2f51e3 2024-08-25T20:26:10.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-08-25T20:26:10.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=5219b813d603b243a6f9cbad1d002fdf 2024-08-25T20:26:10.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-08-25T20:26:10.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary2 2024-08-25T20:26:10.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:10.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:10.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:26:10.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-08-25T20:26:10.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary8 2024-08-25T20:26:10.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:10.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:10.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:26:10.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-08-25T20:26:10.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:26:10.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:26:10.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:10.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:10.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:26:10.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:10.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:10.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:10.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:10.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:10.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:26:10.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-08-25T20:26:10.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:10.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:10.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:26:10.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:10.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:10.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:10.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:10.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:26:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:26:10.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:10.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:10.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:10.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:10.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:10.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:26:10.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:26:10.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:26:10.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:26:10.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-08-25T20:26:10.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:10.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:10.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:26:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:10.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:26:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:26:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 26a31609-06cb-4d50-b283-64d8473a46f9 2024-08-25T20:26:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:10.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:10.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:10.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:10.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:26:10.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:26:10.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: f5ba75f0-8101-4220-854b-b3b32ed57927 2024-08-25T20:26:10.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:10.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:10.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:10.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:10.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:26:11.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:26:11.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:11.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:11.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:26:11.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:11.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:11.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:11.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:11.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:11.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:26:11.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:26:11.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: be5f530f-f7f7-4c25-9be4-a1a854878e85 2024-08-25T20:26:11.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:11.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:11.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:11.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:11.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:11.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:11.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:11.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:26:11.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:26:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-08-25T20:26:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:26:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:11.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:26:11.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:26:11.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:26:11.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:11.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:11.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:26:11.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:11.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:11.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:11.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:26:11.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:11.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:11.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:11.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:11.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:11.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:26:11.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:11.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:26:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:26:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:26:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:11.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:26:11.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:26:11.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7e83313d-862e-4e2d-bacb-d8647c2d63fb 2024-08-25T20:26:11.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:11.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:11.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:11.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:11.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:11.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:11.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:11.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:26:11.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:26:11.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e620d71e-21fd-4188-a15b-b5bbf5db7712 2024-08-25T20:26:11.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:11.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:11.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:11.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:11.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:11.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:11.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:11.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:26:11.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:26:11.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d736788-0ad9-41ef-87c2-7b5bb9179356 2024-08-25T20:26:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:11.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:26:11.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e9b8f42f-7f86-45f5-b7cc-f7349a4ce8de 2024-08-25T20:26:11.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:11.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:11.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:11.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:11.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:11.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:26:11.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:11.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:11.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:26:12.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:26:12.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 699752ba-8a9d-46f9-96ac-1842988c83f7 2024-08-25T20:26:12.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:12.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:12.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:12.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:12.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:12.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:12.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:12.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:26:12.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:26:12.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:12.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:12.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:26:12.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:12.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:12.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:12.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:12.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:12.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:26:12.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:26:12.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 87bb3060-f1c1-405d-a58d-d540f695a6ec 2024-08-25T20:26:12.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:12.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:12.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:12.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:12.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:12.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:12.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:12.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:26:12.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:26:12.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:12.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:12.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:26:12.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:12.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:12.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:12.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:12.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:12.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:26:12.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:26:12.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:12.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:12.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:26:12.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:12.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:12.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:12.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:12.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:12.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:26:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:26:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 26a31609-06cb-4d50-b283-64d8473a46f9 2024-08-25T20:26:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:26:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: f5ba75f0-8101-4220-854b-b3b32ed57927 2024-08-25T20:26:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:26:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:26:13.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:26:13.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:13.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:13.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:26:13.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:13.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:13.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:13.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:13.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:13.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:26:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:26:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:26:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:13.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:13.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:26:13.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:26:13.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:13.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:13.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:26:13.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:13.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:13.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:13.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:13.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:13.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:26:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:26:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:26:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:13.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:13.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:26:14.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:26:14.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7e83313d-862e-4e2d-bacb-d8647c2d63fb 2024-08-25T20:26:14.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:14.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:14.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:14.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:14.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:14.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:14.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:26:14.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e620d71e-21fd-4188-a15b-b5bbf5db7712 2024-08-25T20:26:14.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:14.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:14.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:14.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:14.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:14.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:14.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:26:14.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:14.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:14.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:26:14.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:26:14.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d736788-0ad9-41ef-87c2-7b5bb9179356 2024-08-25T20:26:14.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:14.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:14.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:14.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:14.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:14.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:26:14.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e9b8f42f-7f86-45f5-b7cc-f7349a4ce8de 2024-08-25T20:26:14.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:14.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:14.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:14.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:14.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:26:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:26:14.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:26:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-08-25T20:26:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:26:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:14.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:26:14.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:14.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:14.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:26:14.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:14.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:14.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:14.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:14.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:14.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:26:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:26:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 87bb3060-f1c1-405d-a58d-d540f695a6ec 2024-08-25T20:26:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:14.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:14.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:14.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:26:15.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:26:15.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:15.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:15.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:26:15.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:15.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:15.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:15.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:15.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:15.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:26:15.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:26:15.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: be5f530f-f7f7-4c25-9be4-a1a854878e85 2024-08-25T20:26:15.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:15.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:15.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:15.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:15.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:15.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:15.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:15.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:15.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:26:15.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:15.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:15.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:26:15.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:15.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:15.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:15.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:15.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:15.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:26:15.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:26:15.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dc6102c7-75ee-4e7a-a71e-9ba43c580e44 2024-08-25T20:26:15.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:15.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:15.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:15.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:15.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:15.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:15.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:15.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:26:16.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:26:16.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:16.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:16.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:26:16.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:16.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:16.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:16.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:16.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:16.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:26:16.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:26:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 699752ba-8a9d-46f9-96ac-1842988c83f7 2024-08-25T20:26:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:16.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:16.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:16.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:16.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:26:16.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:16.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:16.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:26:16.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:16.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:16.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:16.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:26:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:26:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:26:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:16.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:16.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:16.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:16.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:16.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:26:17.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:26:17.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 26a31609-06cb-4d50-b283-64d8473a46f9 2024-08-25T20:26:17.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:17.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:17.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:17.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:26:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: f5ba75f0-8101-4220-854b-b3b32ed57927 2024-08-25T20:26:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:17.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:26:17.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:17.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:17.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:26:17.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:17.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:17.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:17.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:17.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:17.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:26:17.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:26:17.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dc6102c7-75ee-4e7a-a71e-9ba43c580e44 2024-08-25T20:26:17.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:17.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:17.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:17.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:17.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:17.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:17.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:17.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:26:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:26:18.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:18.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:26:18.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:26:18.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:18.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:18.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:26:18.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:18.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:18.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:18.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:18.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:18.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:26:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:26:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:26:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:26:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:26:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:18.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:18.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:18.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:26:18.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:18.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:26:18.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:26:18.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e620d71e-21fd-4188-a15b-b5bbf5db7712 2024-08-25T20:26:18.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:18.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:18.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:18.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:18.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:18.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:18.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:18.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:18.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:26:18.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7e83313d-862e-4e2d-bacb-d8647c2d63fb 2024-08-25T20:26:18.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:18.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:18.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:18.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:18.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:18.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:18.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:18.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:18.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:26:18.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e9b8f42f-7f86-45f5-b7cc-f7349a4ce8de 2024-08-25T20:26:18.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:18.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:18.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:18.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:18.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:18.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:26:18.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d736788-0ad9-41ef-87c2-7b5bb9179356 2024-08-25T20:26:18.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:18.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:18.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:18.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:18.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:18.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:18.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:18.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:18.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:26:18.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:18.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:18.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:26:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:26:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:26:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 87bb3060-f1c1-405d-a58d-d540f695a6ec 2024-08-25T20:26:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:18.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:18.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:18.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:21.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:26:21.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:21.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:21.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:26:21.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:21.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:21.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:21.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:21.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:21.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:26:21.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:26:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dc6102c7-75ee-4e7a-a71e-9ba43c580e44 2024-08-25T20:26:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:21.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:21.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:21.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:23.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:26:23.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:23.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:23.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:26:23.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:23.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:23.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:23.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:23.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:23.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:26:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:26:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: be5f530f-f7f7-4c25-9be4-a1a854878e85 2024-08-25T20:26:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:23.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:23.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:23.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:24.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:26:24.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:24.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:24.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:26:24.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:24.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:24.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:24.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:24.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:24.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:26:24.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:26:24.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 699752ba-8a9d-46f9-96ac-1842988c83f7 2024-08-25T20:26:24.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:24.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:24.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:24.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:24.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:24.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:24.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:24.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:26:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:26:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:26:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:26:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:26:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:26:25.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:26:25.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: f5ba75f0-8101-4220-854b-b3b32ed57927 2024-08-25T20:26:25.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:25.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:25.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:25.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:25.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:25.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:26:25.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 26a31609-06cb-4d50-b283-64d8473a46f9 2024-08-25T20:26:25.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:25.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:25.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:25.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:25.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:26.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:26:26.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:26.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:26.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:26:26.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:26.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:26.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:26.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:26.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:26.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:26:26.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:26:26.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:26.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:26.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:26:26.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:26.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:26.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:26.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:26.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:26.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:26:26.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:26:26.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:26.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:26.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:26:26.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:26.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:26.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:26.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:26.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:26.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:26:26.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:26:26.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:26.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:26.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:26:26.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:26.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:26.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:26.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:26.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:26.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:26:26.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:26:26.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7e83313d-862e-4e2d-bacb-d8647c2d63fb 2024-08-25T20:26:26.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:26.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:26.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:26.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:26.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:26.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:26:26.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e620d71e-21fd-4188-a15b-b5bbf5db7712 2024-08-25T20:26:26.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:26.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:26.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:26.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:26.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:26.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:26:26.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e9b8f42f-7f86-45f5-b7cc-f7349a4ce8de 2024-08-25T20:26:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:26.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:26.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:26.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:26.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:26.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:26.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:26.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:26.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:26.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:26:26.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d736788-0ad9-41ef-87c2-7b5bb9179356 2024-08-25T20:26:26.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:26.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:26.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:26.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:26.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:26.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:26.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:26.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:26.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:26:26.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:26.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:26.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:26:26.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:26.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:26.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:26.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:26.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:26.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:26:26.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:26:26.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 87bb3060-f1c1-405d-a58d-d540f695a6ec 2024-08-25T20:26:26.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:26:26.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:26:26.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:26.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:25:58' 2024-08-25T20:26:26.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:26.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:26:26.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:26.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:26:29.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:26:29.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:29.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:29.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:26:29.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:29.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:29.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:29.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:29.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:29.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:26:29.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:26:29.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dc6102c7-75ee-4e7a-a71e-9ba43c580e44 2024-08-25T20:26:29.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:29.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:29.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:29.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:29.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:29.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:29.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:29.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:29.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:29.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:29.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:29.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:29.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:29.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:29.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:29.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:26:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:29.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:29.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:29.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:26:29.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:29.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:26:29.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:29.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:29.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:29.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:29.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:29.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:29.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-08-25T20:26:29.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:29.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:29.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:26:29.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:29.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:29.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:29.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:29.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-08-25T20:26:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:26:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:30.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:30.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:30.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-08-25T20:26:31.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:26:31.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dc6102c7-75ee-4e7a-a71e-9ba43c580e44 2024-08-25T20:26:31.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:31.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:31.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:31.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:31.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:31.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:31.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:31.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:31.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:31.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:31.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:31.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:31.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:31.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:31.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:31.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:26:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:31.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:31.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:31.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:31.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:31.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:31.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:31.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:26:31.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:31.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:26:31.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:31.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:31.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:31.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:31.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:31.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:31.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary2 2024-08-25T20:26:31.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:31.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:31.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:26:31.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:26:31.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-08-25T20:26:31.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:26:31.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:26:31.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:26:31.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-08-25T20:26:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:26:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:26:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:26:31.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd0 2024-08-25T20:26:31.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:26:31.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:26:31.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:26:31.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: be5f530f-f7f7-4c25-9be4-a1a854878e85 2024-08-25T20:26:31.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:31.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:31.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:31.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:31.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:31.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:31.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:31.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:31.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:31.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:31.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:31.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:31.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:31.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:31.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:31.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:31.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:31.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:31.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:31.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:31.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:26:31.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:31.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:31.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:31.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:31.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:31.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:26:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:31.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:26:31.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:31.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:31.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:31.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:31.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:31.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:31.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-08-25T20:26:31.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:31.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:31.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:26:31.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:31.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:31.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:31.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:31.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:26:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:26:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:32.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:32.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:26:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:26:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 699752ba-8a9d-46f9-96ac-1842988c83f7 2024-08-25T20:26:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:32.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:32.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:32.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:32.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:32.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:32.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:32.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:32.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:32.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:32.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:32.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:32.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:32.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:32.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:32.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:32.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:26:32.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:32.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:32.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:32.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:32.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:32.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:32.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:32.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:26:32.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:32.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:26:32.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:32.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:32.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:32.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:32.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:32.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:32.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-08-25T20:26:32.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:32.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:32.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:26:32.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:32.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:32.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:32.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:32.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-08-25T20:26:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:26:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:32.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:32.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:32.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:32.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-08-25T20:26:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:26:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: be5f530f-f7f7-4c25-9be4-a1a854878e85 2024-08-25T20:26:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:32.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:32.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:32.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:32.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:32.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:32.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:32.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:32.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:32.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:26:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:32.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:32.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:32.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:26:32.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:26:32.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:32.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:32.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:32.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary9 2024-08-25T20:26:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:26:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:26:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-08-25T20:26:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:26:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:26:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:26:33.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:33.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:26:33.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:26:33.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 26a31609-06cb-4d50-b283-64d8473a46f9 2024-08-25T20:26:33.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:33.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:33.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:33.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:33.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:33.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:26:33.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: f5ba75f0-8101-4220-854b-b3b32ed57927 2024-08-25T20:26:33.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:33.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:33.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:33.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:33.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:33.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:33.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:33.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:33.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:33.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:33.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:33.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:33.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:33.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:33.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:33.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:33.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:33.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:33.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:33.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:33.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:33.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:33.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:33.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:33.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:33.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:33.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:33.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:33.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:26:33.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:33.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:33.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:26:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:33.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:33.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:33.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:26:33.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:26:33.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:26:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-08-25T20:26:33.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:33.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:33.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:26:33.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:33.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:33.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:33.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:26:33.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:26:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:26:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:33.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-08-25T20:26:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:26:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:33.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:33.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:33.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:33.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-08-25T20:26:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:26:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:33.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd1 2024-08-25T20:26:33.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:26:33.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:26:33.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-08-25T20:26:33.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:33.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:33.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:26:33.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:33.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:33.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:33.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:33.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:33.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-08-25T20:26:33.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:26:33.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 699752ba-8a9d-46f9-96ac-1842988c83f7 2024-08-25T20:26:33.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:33.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:33.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:33.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:33.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:33.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:33.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:33.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:33.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:33.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:33.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:33.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:33.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:33.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:33.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:33.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:33.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:33.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:26:33.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:33.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:33.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:33.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:33.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:33.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:33.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:33.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:26:33.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:33.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:26:33.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:33.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:33.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:33.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:33.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:33.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:33.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary6 2024-08-25T20:26:33.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:33.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:33.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:26:33.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:26:33.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-08-25T20:26:34.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:26:34.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:26:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:26:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:26:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:34.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:34.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:26:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:26:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:26:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:26:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:26:34.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:26:34.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:34.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:26:34.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:34.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:34.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:26:34.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:26:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:26:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:26:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-08-25T20:26:34.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-08-25T20:26:34.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:34.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:26:34.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:34.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:34.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-08-25T20:26:34.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-08-25T20:26:34.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:34.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:26:34.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:34.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:34.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-08-25T20:26:34.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:26:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7e83313d-862e-4e2d-bacb-d8647c2d63fb 2024-08-25T20:26:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:26:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e9b8f42f-7f86-45f5-b7cc-f7349a4ce8de 2024-08-25T20:26:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:34.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:34.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:26:34.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e620d71e-21fd-4188-a15b-b5bbf5db7712 2024-08-25T20:26:34.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:34.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:34.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:34.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:34.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:34.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:34.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:34.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:34.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:34.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:34.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:34.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:34.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:34.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:34.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:34.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:34.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:34.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:34.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:26:34.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d736788-0ad9-41ef-87c2-7b5bb9179356 2024-08-25T20:26:34.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:34.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:34.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:34.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:34.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:34.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:34.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:34.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:34.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:34.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:34.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:34.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:34.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:34.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:34.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:26:34.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:34.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:34.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:34.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:34.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:34.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:34.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:34.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:34.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:26:34.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:34.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:26:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:34.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:34.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:34.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:34.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:34.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:26:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:26:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: f5ba75f0-8101-4220-854b-b3b32ed57927 2024-08-25T20:26:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:34.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:26:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 26a31609-06cb-4d50-b283-64d8473a46f9 2024-08-25T20:26:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:34.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:34.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:34.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:34.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:34.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:34.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:34.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:34.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:34.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:34.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:34.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:34.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:34.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:34.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:34.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:34.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:34.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:34.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:26:34.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:34.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:34.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:34.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:34.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:26:34.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:34.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:34.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd2 2024-08-25T20:26:34.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:26:34.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:26:34.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:34.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:34.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:26:34.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:26:34.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:34.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-08-25T20:26:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:26:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:34.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:34.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:34.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:34.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:34.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:34.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:26:34.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:34.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:34.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:26:34.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:26:34.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:26:34.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:34.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:34.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:34.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:26:34.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-08-25T20:26:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:26:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:34.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:34.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:34.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:34.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:26:34.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:34.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-08-25T20:26:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:26:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:34.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:34.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:34.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:34.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-08-25T20:26:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:26:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:26:34.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:34.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:34.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:26:34.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:26:34.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:26:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:26:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:26:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:26:34.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:34.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary10 2024-08-25T20:26:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:26:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:26:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-08-25T20:26:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:34.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:34.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:34.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary5 2024-08-25T20:26:34.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:34.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:26:34.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:26:34.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-08-25T20:26:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:26:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 87bb3060-f1c1-405d-a58d-d540f695a6ec 2024-08-25T20:26:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:34.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:34.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:34.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:34.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:34.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:34.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:34.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:34.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:34.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:34.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:34.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:34.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:34.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:34.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:26:34.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:34.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:26:34.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:34.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:34.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:34.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:34.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:35.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:35.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:35.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:26:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:35.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:26:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:35.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:35.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:35.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:35.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-08-25T20:26:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:26:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:26:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:26:35.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:26:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:26:35.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:26:35.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:26:35.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-08-25T20:26:35.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:26:35.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:26:35.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-08-25T20:26:35.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd3 2024-08-25T20:26:35.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:26:35.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:26:35.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd4 2024-08-25T20:26:35.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:26:35.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:26:35.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-08-25T20:26:35.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:35.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:35.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:26:35.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:35.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:35.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:35.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:35.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:35.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-08-25T20:26:35.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-08-25T20:26:35.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:35.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:35.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:26:35.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:35.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:35.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:35.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:35.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:35.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-08-25T20:26:35.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-08-25T20:26:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:26:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-08-25T20:26:35.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-08-25T20:26:35.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:35.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:35.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:26:35.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:35.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:35.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:35.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:35.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:35.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-08-25T20:26:35.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:26:35.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e9b8f42f-7f86-45f5-b7cc-f7349a4ce8de 2024-08-25T20:26:35.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:35.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:35.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:35.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:35.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:35.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:35.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:35.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:35.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:35.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:26:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e620d71e-21fd-4188-a15b-b5bbf5db7712 2024-08-25T20:26:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:35.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:35.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:35.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:35.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:35.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:35.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:35.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:35.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:26:35.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:35.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:35.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:35.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:35.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:35.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:35.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:35.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:26:35.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7e83313d-862e-4e2d-bacb-d8647c2d63fb 2024-08-25T20:26:35.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:35.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:35.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:35.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:35.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:35.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:35.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:35.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:26:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d736788-0ad9-41ef-87c2-7b5bb9179356 2024-08-25T20:26:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:35.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:35.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:35.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:35.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:26:35.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:35.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:35.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:35.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:35.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:35.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:35.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:35.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:35.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:35.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:35.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:35.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:35.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:35.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:35.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:35.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:35.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:35.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:35.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:35.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:35.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:35.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:35.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:26:35.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:35.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:35.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:35.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:35.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:35.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:35.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:35.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:35.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:35.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:35.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:35.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:26:35.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:35.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:35.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:35.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:35.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:36.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-08-25T20:26:36.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:36.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:36.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:26:36.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:36.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:26:36.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:26:36.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:36.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:36.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-08-25T20:26:36.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:36.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:36.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:26:36.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:36.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:26:36.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:36.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:36.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:36.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:36.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:36.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:36.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary7 2024-08-25T20:26:36.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:36.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:36.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:26:36.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:26:36.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-08-25T20:26:36.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:36.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:36.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:26:36.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:36.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:26:36.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:36.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:36.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:36.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:36.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:36.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:36.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary4 2024-08-25T20:26:36.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:36.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:36.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:26:36.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:26:36.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-08-25T20:26:36.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:26:36.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 87bb3060-f1c1-405d-a58d-d540f695a6ec 2024-08-25T20:26:36.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:26:36.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:26:36.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:26:36.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:26:28' 2024-08-25T20:26:36.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:26:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:26:36.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:26:36.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:26:36.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:26:36.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:26:36.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:36.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:36.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:36.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:26:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:26:36.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:26:36.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:26:36.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:26:36.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:36.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:26:36.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:36.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:36.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:36.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-08-25 20:26:28' 2024-08-25T20:26:36.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:26:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:26:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:26:28' 2024-08-25T20:26:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:26:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:26:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:26:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:36.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:36.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:26:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary3 2024-08-25T20:26:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:26:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:26:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-08-25T20:26:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:36.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:26:36.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:36.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:36.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:36.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:36.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary1 2024-08-25T20:26:36.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:36.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:36.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:26:36.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:26:36.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-08-25T20:26:36.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:26:28' 2024-08-25T20:26:36.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:26:36.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:26:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:26:28' 2024-08-25T20:26:36.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:26:36.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:26:36.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:26:36.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:26:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:26:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:26:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary8 2024-08-25T20:26:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:26:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:26:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:26:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:26:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-08-25T20:26:37.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:26:37.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:26:37.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:26:37.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:26:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:26:37.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:26:37.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:26:37.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-08-25T20:26:37.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:26:37.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:26:37.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-08-25T20:26:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:26:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:26:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:26:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:26:37.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-08-25T20:26:37.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-08-25T20:26:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:26:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:26:37.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-08-25T20:26:37.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd7 2024-08-25T20:26:37.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd5 2024-08-25T20:26:37.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:26:37.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd6 2024-08-25T20:26:37.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:26:37.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd8 2024-08-25T20:26:37.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:26:37.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:26:37.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:26:37.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:26:37.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:26:37.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:26:37.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd9 2024-08-25T20:26:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:26:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:27:21.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=e31838f8dca8c47c076c06c9b2192cc8 2024-08-25T20:27:21.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-08-25T20:27:22.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ e31838f8dca8c47c076c06c9b2192cc8 != \e\3\1\8\3\8\f\8\d\c\a\8\c\4\7\c\0\7\6\c\0\6\c\9\b\2\1\9\2\c\c\8 ]] 2024-08-25T20:27:24.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=2775cbd0a44ee877a50b5d89fa2f51e3 2024-08-25T20:27:24.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-08-25T20:27:24.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 2775cbd0a44ee877a50b5d89fa2f51e3 != \2\7\7\5\c\b\d\0\a\4\4\e\e\8\7\7\a\5\0\b\5\d\8\9\f\a\2\f\5\1\e\3 ]] 2024-08-25T20:27:25.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=e267dfc07734ad1fea743b1ab55d1f60 2024-08-25T20:27:25.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-08-25T20:27:26.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ e267dfc07734ad1fea743b1ab55d1f60 != \e\2\6\7\d\f\c\0\7\7\3\4\a\d\1\f\e\a\7\4\3\b\1\a\b\5\5\d\1\f\6\0 ]] 2024-08-25T20:27:29.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=cdd2aad006dc5e72b0a9921228e29eb4 2024-08-25T20:27:29.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-08-25T20:27:29.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=404de2bf7083c6cbb81a334f9a25661b 2024-08-25T20:27:29.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-08-25T20:27:29.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ cdd2aad006dc5e72b0a9921228e29eb4 != \c\d\d\2\a\a\d\0\0\6\d\c\5\e\7\2\b\0\a\9\9\2\1\2\2\8\e\2\9\e\b\4 ]] 2024-08-25T20:27:29.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=0032916ffb088c625320414bc8abd842 2024-08-25T20:27:29.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-08-25T20:27:29.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 404de2bf7083c6cbb81a334f9a25661b != \4\0\4\d\e\2\b\f\7\0\8\3\c\6\c\b\b\8\1\a\3\3\4\f\9\a\2\5\6\6\1\b ]] 2024-08-25T20:27:29.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 0032916ffb088c625320414bc8abd842 != \0\0\3\2\9\1\6\f\f\b\0\8\8\c\6\2\5\3\2\0\4\1\4\b\c\8\a\b\d\8\4\2 ]] 2024-08-25T20:27:29.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:27:29.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 78269 2024-08-25T20:27:29.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:27:29.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 78271 2024-08-25T20:27:30.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=34a43060c745963fa84ea70b378a5119 2024-08-25T20:27:30.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-08-25T20:27:30.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=3afd7f865d4a38ef3c819b723fddc2e2 2024-08-25T20:27:30.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-08-25T20:27:30.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 34a43060c745963fa84ea70b378a5119 != \3\4\a\4\3\0\6\0\c\7\4\5\9\6\3\f\a\8\4\e\a\7\0\b\3\7\8\a\5\1\1\9 ]] 2024-08-25T20:27:30.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:27:30.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 78273 2024-08-25T20:27:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=5219b813d603b243a6f9cbad1d002fdf 2024-08-25T20:27:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-08-25T20:27:30.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 3afd7f865d4a38ef3c819b723fddc2e2 != \3\a\f\d\7\f\8\6\5\d\4\a\3\8\e\f\3\c\8\1\9\b\7\2\3\f\d\d\c\2\e\2 ]] 2024-08-25T20:27:30.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 5219b813d603b243a6f9cbad1d002fdf != \5\2\1\9\b\8\1\3\d\6\0\3\b\2\4\3\a\6\f\9\c\b\a\d\1\d\0\0\2\f\d\f ]] 2024-08-25T20:27:31.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=1cde6aaa698efd5a71cfe6bc2dba2ede 2024-08-25T20:27:31.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-08-25T20:27:32.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 1cde6aaa698efd5a71cfe6bc2dba2ede != \1\c\d\e\6\a\a\a\6\9\8\e\f\d\5\a\7\1\c\f\e\6\b\c\2\d\b\a\2\e\d\e ]] 2024-08-25T20:27:32.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:27:32.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 78275 2024-08-25T20:27:32.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:27:32.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 78277 2024-08-25T20:27:32.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:27:32.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 78279 2024-08-25T20:27:32.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 78281 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 78283 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 78285 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:27:32.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-08-25T20:27:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:27:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:32.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:27:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-08-25T20:27:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:27:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:27:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:27:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d736788-0ad9-41ef-87c2-7b5bb9179356 2024-08-25T20:27:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:27:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3425484.8,"bytes_per_snapshot":68509696.0,"last_snapshot_bytes":68509696,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"} 2024-08-25T20:27:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:27:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:27:28' 2024-08-25T20:27:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:27:33.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:27:33.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:27:33.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3425484.8,"bytes_per_snapshot":68509696.0,"last_snapshot_bytes":68509696,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"} 2024-08-25T20:27:33.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:27:33.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:33.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:27:33.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:33.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:27:33.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:27:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:27:33.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:27:33.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:33.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-08-25 20:27:28' 2024-08-25T20:27:33.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:33.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:33.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:27:33.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:33.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:27:28' 2024-08-25T20:27:33.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:33.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:33.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:27:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary14d736788-0ad9-41ef-87c2-7b5bb9179356up+replayingreplaying, {"bytes_per_second":3425484.8,"bytes_per_snapshot":68509696.0,"last_snapshot_bytes":68509696,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+replayingreplaying, {"bytes_per_second":3224371.2,"bytes_per_snapshot":64487424.0,"last_snapshot_bytes":64487424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+replayingreplaying, {"bytes_per_second":3180544.0,"bytes_per_snapshot":63610880.0,"last_snapshot_bytes":63610880,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617591,"remote_snapshot_timestamp":1724617591,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+replayingreplaying, {"bytes_per_second":3396812.8,"bytes_per_snapshot":67936256.0,"last_snapshot_bytes":67936256,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+replayingreplaying, {"bytes_per_second":3427328.0,"bytes_per_snapshot":68546560.0,"last_snapshot_bytes":68546560,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+replayingreplaying, {"bytes_per_second":3391488.0,"bytes_per_snapshot":67829760.0,"last_snapshot_bytes":67829760,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+replayingreplaying, {"bytes_per_second":3424051.2,"bytes_per_snapshot":68481024.0,"last_snapshot_bytes":68481024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617594,"remote_snapshot_timestamp":1724617594,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+replayingreplaying, {"bytes_per_second":3319193.6,"bytes_per_snapshot":66383872.0,"last_snapshot_bytes":66383872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+replayingreplaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+replayingreplaying, {"bytes_per_second":3588915.2,"bytes_per_snapshot":71778304.0,"last_snapshot_bytes":71778304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617593,"remote_snapshot_timestamp":1724617593,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28' 2024-08-25T20:27:33.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:27:33.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:27:33.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:33.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:27:33.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:27:33.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:33.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:27:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:33.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:33.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:33.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary1 2024-08-25T20:27:33.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:27:33.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:33.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:27:33.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-08-25T20:27:33.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-08-25T20:27:33.544 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:33.542+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3a3e680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:27:33.544 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:33.542+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973680 [3/4d736788-0ad9-41ef-87c2-7b5bb9179356] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:27:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:27:35.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-08-25T20:27:35.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:35.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:35.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:27:35.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:35.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:27:35.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary1 2024-08-25T20:27:35.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:35.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary1 2024-08-25T20:27:35.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:27:35.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:27:35.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-08-25T20:27:35.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:35.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:35.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:27:35.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:27:35.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-08-25T20:27:35.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:35.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:35.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:27:35.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:35.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:35.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:35.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:35.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:27:36.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-08-25T20:27:36.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:36.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:36.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:27:36.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:36.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:36.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:36.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:36.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:36.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:27:36.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:27:36.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: dc6102c7-75ee-4e7a-a71e-9ba43c580e44 2024-08-25T20:27:36.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:27:36.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3180544.0,"bytes_per_snapshot":63610880.0,"last_snapshot_bytes":63610880,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617591,"remote_snapshot_timestamp":1724617591,"replay_state":"idle"} 2024-08-25T20:27:36.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:27:36.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:27:28' 2024-08-25T20:27:36.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:27:36.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:27:36.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:27:36.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3180544.0,"bytes_per_snapshot":63610880.0,"last_snapshot_bytes":63610880,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617591,"remote_snapshot_timestamp":1724617591,"replay_state":"idle"} 2024-08-25T20:27:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:27:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:27:36.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:36.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:27:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:27:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:27:36.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:27:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-08-25 20:27:28' 2024-08-25T20:27:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:27:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:27:28' 2024-08-25T20:27:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:27:36.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING941904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+replayingreplaying, {"bytes_per_second":3224371.2,"bytes_per_snapshot":64487424.0,"last_snapshot_bytes":64487424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary2dc6102c7-75ee-4e7a-a71e-9ba43c580e44up+replayingreplaying, {"bytes_per_second":3180544.0,"bytes_per_snapshot":63610880.0,"last_snapshot_bytes":63610880,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617591,"remote_snapshot_timestamp":1724617591,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+replayingreplaying, {"bytes_per_second":3396812.8,"bytes_per_snapshot":67936256.0,"last_snapshot_bytes":67936256,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+replayingreplaying, {"bytes_per_second":3427328.0,"bytes_per_snapshot":68546560.0,"last_snapshot_bytes":68546560,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+replayingreplaying, {"bytes_per_second":3391488.0,"bytes_per_snapshot":67829760.0,"last_snapshot_bytes":67829760,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+replayingreplaying, {"bytes_per_second":3424051.2,"bytes_per_snapshot":68481024.0,"last_snapshot_bytes":68481024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617594,"remote_snapshot_timestamp":1724617594,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+replayingreplaying, {"bytes_per_second":3319193.6,"bytes_per_snapshot":66383872.0,"last_snapshot_bytes":66383872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+replayingreplaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+replayingreplaying, {"bytes_per_second":3588915.2,"bytes_per_snapshot":71778304.0,"last_snapshot_bytes":71778304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617593,"remote_snapshot_timestamp":1724617593,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28' 2024-08-25T20:27:36.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:27:36.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:27:36.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:36.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:27:36.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:27:36.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:36.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:27:36.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:36.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:36.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:36.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary2 2024-08-25T20:27:36.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:27:36.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:36.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:27:36.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-08-25T20:27:37.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-08-25T20:27:37.166 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:37.164+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3662000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:27:37.167 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:37.164+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972500 [3/dc6102c7-75ee-4e7a-a71e-9ba43c580e44] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:27:38.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:27:38.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-08-25T20:27:38.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:38.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:38.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:27:38.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:38.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:27:38.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary2 2024-08-25T20:27:38.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:38.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary2 2024-08-25T20:27:38.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:27:38.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:27:38.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-08-25T20:27:38.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:38.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:38.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:27:38.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:27:38.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-08-25T20:27:38.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:38.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:38.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:27:38.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:38.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:38.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:38.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:38.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:27:39.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-08-25T20:27:39.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:39.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:39.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:27:39.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:39.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:39.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:39.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:39.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:39.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:27:39.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:27:39.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7e83313d-862e-4e2d-bacb-d8647c2d63fb 2024-08-25T20:27:39.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:27:39.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3396812.8,"bytes_per_snapshot":67936256.0,"last_snapshot_bytes":67936256,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"} 2024-08-25T20:27:39.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:27:39.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:27:28' 2024-08-25T20:27:39.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:27:39.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:27:39.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:27:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3396812.8,"bytes_per_snapshot":67936256.0,"last_snapshot_bytes":67936256,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"} 2024-08-25T20:27:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:27:39.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:39.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:27:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:39.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:27:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:27:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:27:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:27:39.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:39.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-08-25 20:27:28' 2024-08-25T20:27:39.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:39.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:39.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:27:39.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:39.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:27:28' 2024-08-25T20:27:39.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:39.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:39.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:27:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING841904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+replayingreplaying, {"bytes_per_second":3224371.2,"bytes_per_snapshot":64487424.0,"last_snapshot_bytes":64487424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary37e83313d-862e-4e2d-bacb-d8647c2d63fbup+replayingreplaying, {"bytes_per_second":3396812.8,"bytes_per_snapshot":67936256.0,"last_snapshot_bytes":67936256,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+replayingreplaying, {"bytes_per_second":3427328.0,"bytes_per_snapshot":68546560.0,"last_snapshot_bytes":68546560,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+replayingreplaying, {"bytes_per_second":3391488.0,"bytes_per_snapshot":67829760.0,"last_snapshot_bytes":67829760,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+replayingreplaying, {"bytes_per_second":3424051.2,"bytes_per_snapshot":68481024.0,"last_snapshot_bytes":68481024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617594,"remote_snapshot_timestamp":1724617594,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+replayingreplaying, {"bytes_per_second":3319193.6,"bytes_per_snapshot":66383872.0,"last_snapshot_bytes":66383872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+replayingreplaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+replayingreplaying, {"bytes_per_second":3588915.2,"bytes_per_snapshot":71778304.0,"last_snapshot_bytes":71778304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617593,"remote_snapshot_timestamp":1724617593,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28' 2024-08-25T20:27:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:27:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:27:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:27:39.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:27:39.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:39.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:27:39.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:39.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:39.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:39.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary3 2024-08-25T20:27:39.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:27:39.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:39.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:27:39.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-08-25T20:27:39.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-08-25T20:27:40.014 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:40.012+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8ae1a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:27:40.014 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:40.012+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ae4fe2a00 [3/7e83313d-862e-4e2d-bacb-d8647c2d63fb] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:27:41.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:27:41.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-08-25T20:27:41.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:41.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:41.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:27:41.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:41.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:27:41.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary3 2024-08-25T20:27:41.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:41.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary3 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:41.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:41.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:41.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:27:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-08-25T20:27:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:27:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:42.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:42.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:27:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:27:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e620d71e-21fd-4188-a15b-b5bbf5db7712 2024-08-25T20:27:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:27:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3427328.0,"bytes_per_snapshot":68546560.0,"last_snapshot_bytes":68546560,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"} 2024-08-25T20:27:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:27:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:27:28' 2024-08-25T20:27:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:27:42.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:27:42.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:27:42.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3427328.0,"bytes_per_snapshot":68546560.0,"last_snapshot_bytes":68546560,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"} 2024-08-25T20:27:42.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:27:42.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:42.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:27:42.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:42.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:27:42.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:27:42.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:27:42.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:27:42.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:42.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-08-25 20:27:28' 2024-08-25T20:27:42.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:42.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:42.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:27:42.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:42.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:27:28' 2024-08-25T20:27:42.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:42.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:42.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:27:42.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING741904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+replayingreplaying, {"bytes_per_second":3224371.2,"bytes_per_snapshot":64487424.0,"last_snapshot_bytes":64487424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary4e620d71e-21fd-4188-a15b-b5bbf5db7712up+replayingreplaying, {"bytes_per_second":3427328.0,"bytes_per_snapshot":68546560.0,"last_snapshot_bytes":68546560,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+replayingreplaying, {"bytes_per_second":3391488.0,"bytes_per_snapshot":67829760.0,"last_snapshot_bytes":67829760,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+replayingreplaying, {"bytes_per_second":3424051.2,"bytes_per_snapshot":68481024.0,"last_snapshot_bytes":68481024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617594,"remote_snapshot_timestamp":1724617594,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+replayingreplaying, {"bytes_per_second":3319193.6,"bytes_per_snapshot":66383872.0,"last_snapshot_bytes":66383872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+replayingreplaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+replayingreplaying, {"bytes_per_second":3588915.2,"bytes_per_snapshot":71778304.0,"last_snapshot_bytes":71778304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617593,"remote_snapshot_timestamp":1724617593,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28' 2024-08-25T20:27:42.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:27:42.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:27:42.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:42.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:27:42.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:27:42.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:42.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:27:42.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:42.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:42.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:42.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary4 2024-08-25T20:27:42.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:27:42.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:42.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:27:42.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-08-25T20:27:42.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-08-25T20:27:42.870 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:42.869+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade2d8d00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:27:42.870 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:42.869+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973180 [3/e620d71e-21fd-4188-a15b-b5bbf5db7712] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:27:44.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:27:44.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-08-25T20:27:44.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:44.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:44.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:27:44.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:44.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:27:44.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary4 2024-08-25T20:27:44.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:44.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary4 2024-08-25T20:27:44.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:44.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:44.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:27:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-08-25T20:27:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:27:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:27:45.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:27:45.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: f5ba75f0-8101-4220-854b-b3b32ed57927 2024-08-25T20:27:45.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:27:45.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3391488.0,"bytes_per_snapshot":67829760.0,"last_snapshot_bytes":67829760,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"} 2024-08-25T20:27:45.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:27:45.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:27:28' 2024-08-25T20:27:45.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:27:45.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:27:45.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:27:45.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3391488.0,"bytes_per_snapshot":67829760.0,"last_snapshot_bytes":67829760,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"} 2024-08-25T20:27:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:27:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:27:45.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:45.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:27:45.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:27:45.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:27:45.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:27:45.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:45.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-08-25 20:27:28' 2024-08-25T20:27:45.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:45.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:45.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:27:45.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:45.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:27:28' 2024-08-25T20:27:45.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:45.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:45.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:27:45.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING641904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+replayingreplaying, {"bytes_per_second":3224371.2,"bytes_per_snapshot":64487424.0,"last_snapshot_bytes":64487424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary5f5ba75f0-8101-4220-854b-b3b32ed57927up+replayingreplaying, {"bytes_per_second":3391488.0,"bytes_per_snapshot":67829760.0,"last_snapshot_bytes":67829760,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+replayingreplaying, {"bytes_per_second":3424051.2,"bytes_per_snapshot":68481024.0,"last_snapshot_bytes":68481024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617594,"remote_snapshot_timestamp":1724617594,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+replayingreplaying, {"bytes_per_second":3319193.6,"bytes_per_snapshot":66383872.0,"last_snapshot_bytes":66383872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+replayingreplaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+replayingreplaying, {"bytes_per_second":3588915.2,"bytes_per_snapshot":71778304.0,"last_snapshot_bytes":71778304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617593,"remote_snapshot_timestamp":1724617593,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28' 2024-08-25T20:27:45.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:27:45.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:27:45.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:45.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:27:45.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:27:45.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:45.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:27:45.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary5 2024-08-25T20:27:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:27:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:27:45.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-08-25T20:27:45.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-08-25T20:27:45.895 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:45.894+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adf71f520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:27:45.895 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:45.894+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972780 [3/f5ba75f0-8101-4220-854b-b3b32ed57927] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:27:47.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:27:47.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-08-25T20:27:47.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:47.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:47.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:27:47.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:47.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:27:47.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary5 2024-08-25T20:27:47.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:47.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary5 2024-08-25T20:27:47.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:27:47.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:27:48.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-08-25T20:27:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:27:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:27:48.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:27:48.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 699752ba-8a9d-46f9-96ac-1842988c83f7 2024-08-25T20:27:48.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:27:48.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3424051.2,"bytes_per_snapshot":68481024.0,"last_snapshot_bytes":68481024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617594,"remote_snapshot_timestamp":1724617594,"replay_state":"idle"} 2024-08-25T20:27:48.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:27:48.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:27:28' 2024-08-25T20:27:48.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:27:48.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:27:48.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:27:48.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3424051.2,"bytes_per_snapshot":68481024.0,"last_snapshot_bytes":68481024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617594,"remote_snapshot_timestamp":1724617594,"replay_state":"idle"} 2024-08-25T20:27:48.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:27:48.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:48.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:27:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:27:48.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:27:48.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:27:48.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:27:48.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:48.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-08-25 20:27:28' 2024-08-25T20:27:48.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:48.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:48.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:27:48.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:48.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:27:28' 2024-08-25T20:27:48.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:48.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:48.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:27:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING541904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+replayingreplaying, {"bytes_per_second":3224371.2,"bytes_per_snapshot":64487424.0,"last_snapshot_bytes":64487424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary6699752ba-8a9d-46f9-96ac-1842988c83f7up+replayingreplaying, {"bytes_per_second":3424051.2,"bytes_per_snapshot":68481024.0,"last_snapshot_bytes":68481024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617594,"remote_snapshot_timestamp":1724617594,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+replayingreplaying, {"bytes_per_second":3319193.6,"bytes_per_snapshot":66383872.0,"last_snapshot_bytes":66383872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+replayingreplaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+replayingreplaying, {"bytes_per_second":3588915.2,"bytes_per_snapshot":71778304.0,"last_snapshot_bytes":71778304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617593,"remote_snapshot_timestamp":1724617593,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28' 2024-08-25T20:27:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:27:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:27:48.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:48.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:27:48.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:27:48.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:48.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:27:48.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:48.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:48.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:48.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary6 2024-08-25T20:27:48.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:27:48.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:48.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:27:48.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-08-25T20:27:48.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-08-25T20:27:48.871 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:48.871+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3454b60 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:27:48.871 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:48.871+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972f00 [3/699752ba-8a9d-46f9-96ac-1842988c83f7] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:27:50.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:27:50.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-08-25T20:27:50.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:50.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:50.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:27:50.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:50.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:27:50.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary6 2024-08-25T20:27:50.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:50.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary6 2024-08-25T20:27:50.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:27:50.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:50.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:27:51.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-08-25T20:27:51.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:51.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:51.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:27:51.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:51.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:51.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:51.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:51.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:51.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:27:51.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:27:51.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e9b8f42f-7f86-45f5-b7cc-f7349a4ce8de 2024-08-25T20:27:51.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:27:51.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3319193.6,"bytes_per_snapshot":66383872.0,"last_snapshot_bytes":66383872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"} 2024-08-25T20:27:51.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:27:51.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:27:28' 2024-08-25T20:27:51.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:27:51.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:27:51.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:27:51.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3319193.6,"bytes_per_snapshot":66383872.0,"last_snapshot_bytes":66383872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"} 2024-08-25T20:27:51.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:27:51.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:51.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:27:51.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:51.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:27:51.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:27:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:27:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:27:51.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:51.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-08-25 20:27:28' 2024-08-25T20:27:51.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:51.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:51.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:27:51.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:51.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:27:28' 2024-08-25T20:27:51.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:51.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:51.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:27:51.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING441904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+replayingreplaying, {"bytes_per_second":3224371.2,"bytes_per_snapshot":64487424.0,"last_snapshot_bytes":64487424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary7e9b8f42f-7f86-45f5-b7cc-f7349a4ce8deup+replayingreplaying, {"bytes_per_second":3319193.6,"bytes_per_snapshot":66383872.0,"last_snapshot_bytes":66383872,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+replayingreplaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+replayingreplaying, {"bytes_per_second":3588915.2,"bytes_per_snapshot":71778304.0,"last_snapshot_bytes":71778304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617593,"remote_snapshot_timestamp":1724617593,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28' 2024-08-25T20:27:51.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:27:51.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:27:51.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:51.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:27:51.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:27:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:27:51.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary7 2024-08-25T20:27:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:27:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:27:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-08-25T20:27:51.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-08-25T20:27:51.865 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:51.864+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8ac9c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:27:51.865 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:51.864+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972000 [3/e9b8f42f-7f86-45f5-b7cc-f7349a4ce8de] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:27:53.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:27:53.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-08-25T20:27:53.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:53.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:53.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:27:53.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:53.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:27:53.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary7 2024-08-25T20:27:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary7 2024-08-25T20:27:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:27:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:27:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-08-25T20:27:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:27:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:27:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-08-25T20:27:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:27:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:27:54.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-08-25T20:27:54.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:54.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:54.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:27:54.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:54.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:54.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:54.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:54.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:54.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:27:54.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:27:54.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 87bb3060-f1c1-405d-a58d-d540f695a6ec 2024-08-25T20:27:54.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:27:54.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"} 2024-08-25T20:27:54.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:27:54.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:27:28' 2024-08-25T20:27:54.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:27:54.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:27:54.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:27:54.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"} 2024-08-25T20:27:54.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:27:54.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:54.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:27:54.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:54.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:27:54.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:27:54.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:27:54.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:27:54.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:54.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-08-25 20:27:28' 2024-08-25T20:27:54.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:54.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:54.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:27:54.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:54.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:27:28' 2024-08-25T20:27:54.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:54.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:54.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:27:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING341904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+replayingreplaying, {"bytes_per_second":3224371.2,"bytes_per_snapshot":64487424.0,"last_snapshot_bytes":64487424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary887bb3060-f1c1-405d-a58d-d540f695a6ecup+replayingreplaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617597,"remote_snapshot_timestamp":1724617597,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+replayingreplaying, {"bytes_per_second":3588915.2,"bytes_per_snapshot":71778304.0,"last_snapshot_bytes":71778304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617593,"remote_snapshot_timestamp":1724617593,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28' 2024-08-25T20:27:54.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:27:54.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:27:54.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:54.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:27:54.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:27:54.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:54.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:27:54.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary8 2024-08-25T20:27:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:27:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:27:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-08-25T20:27:54.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-08-25T20:27:54.901 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:54.901+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3455d40 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:27:54.901 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:54.901+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973b80 [3/87bb3060-f1c1-405d-a58d-d540f695a6ec] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:27:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:27:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-08-25T20:27:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:27:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:56.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:27:56.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary8 2024-08-25T20:27:56.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:56.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary8 2024-08-25T20:27:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:27:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:27:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-08-25T20:27:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:27:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:27:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-08-25T20:27:56.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:56.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:56.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:27:56.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:56.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:56.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:56.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:56.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:27:57.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-08-25T20:27:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:27:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:27:57.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:27:57.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: be5f530f-f7f7-4c25-9be4-a1a854878e85 2024-08-25T20:27:57.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:27:57.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3588915.2,"bytes_per_snapshot":71778304.0,"last_snapshot_bytes":71778304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617593,"remote_snapshot_timestamp":1724617593,"replay_state":"idle"} 2024-08-25T20:27:57.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:27:57.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:27:28' 2024-08-25T20:27:57.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:27:57.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:27:57.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:27:57.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3588915.2,"bytes_per_snapshot":71778304.0,"last_snapshot_bytes":71778304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617593,"remote_snapshot_timestamp":1724617593,"replay_state":"idle"} 2024-08-25T20:27:57.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:27:57.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:27:57.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:57.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:27:57.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:27:57.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:27:57.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:27:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-08-25 20:27:28' 2024-08-25T20:27:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:27:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:27:28' 2024-08-25T20:27:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:27:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:27:57.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:27:57.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING241904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+replayingreplaying, {"bytes_per_second":3224371.2,"bytes_per_snapshot":64487424.0,"last_snapshot_bytes":64487424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28image-primary9be5f530f-f7f7-4c25-9be4-a1a854878e85up+replayingreplaying, {"bytes_per_second":3588915.2,"bytes_per_snapshot":71778304.0,"last_snapshot_bytes":71778304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617593,"remote_snapshot_timestamp":1724617593,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:28' 2024-08-25T20:27:57.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:27:57.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:27:57.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:28' 2024-08-25T20:27:57.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:27:57.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:27:57.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:27:57.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:27:57.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:27:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary9 2024-08-25T20:27:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:27:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:27:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-08-25T20:27:57.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-08-25T20:27:57.844 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:57.843+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adfb669c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:27:57.844 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:57.843+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ae365fb80 [3/be5f530f-f7f7-4c25-9be4-a1a854878e85] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:27:58.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:58.331+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4e691e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:27:58.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:27:58.331+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ae365fb80 [3/be5f530f-f7f7-4c25-9be4-a1a854878e85] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:27:59.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:27:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-08-25T20:27:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:27:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:27:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary9 2024-08-25T20:27:59.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:27:59.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary9 2024-08-25T20:27:59.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:27:59.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:27:59.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-08-25T20:27:59.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:59.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:59.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:27:59.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:27:59.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-08-25T20:27:59.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:27:59.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:27:59.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:27:59.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:27:59.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:27:59.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:27:59.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:27:59.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:28:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-08-25T20:28:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:28:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:28:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:28:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:28:00.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:28:00.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:28:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:28:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 26a31609-06cb-4d50-b283-64d8473a46f9 2024-08-25T20:28:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:28:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":64487424.0,"last_snapshot_bytes":64487424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"} 2024-08-25T20:28:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:28:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:27:58' 2024-08-25T20:28:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:28:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:28:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:28:00.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":64487424.0,"last_snapshot_bytes":64487424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"} 2024-08-25T20:28:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:28:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:28:00.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:28:00.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:28:00.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:28:00.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:28:00.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:28:00.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:28:00.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:58' 2024-08-25T20:28:00.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-08-25 20:27:58' 2024-08-25T20:28:00.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:28:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:27:58' 2024-08-25T20:28:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:28:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:28:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:28:00.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING141904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1026a31609-06cb-4d50-b283-64d8473a46f9up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":64487424.0,"last_snapshot_bytes":64487424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724617595,"remote_snapshot_timestamp":1724617595,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:27:58' 2024-08-25T20:28:00.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:28:00.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:28:00.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:27:58' 2024-08-25T20:28:00.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:28:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:28:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:28:00.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:28:00.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:28:00.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:00.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:00.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary10 2024-08-25T20:28:00.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:28:00.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:00.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:00.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-08-25T20:28:00.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-08-25T20:28:00.855 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:28:00.854+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4e6d040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:28:00.855 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:28:00.854+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972c80 [3/26a31609-06cb-4d50-b283-64d8473a46f9] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:28:02.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:28:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-08-25T20:28:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:28:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:28:02.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary10 2024-08-25T20:28:02.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:02.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary10 2024-08-25T20:28:02.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..10} 2024-08-25T20:28:02.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS=() 2024-08-25T20:28:02.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS=() 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS=() 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS=() 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RET=0 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary1 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:02.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-08-25T20:28:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:28:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:28:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-08-25T20:28:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-08-25T20:28:03.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:28:03.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-08-25T20:28:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary1': 2024-08-25T20:28:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:28:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:28:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:28:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 29c274ec4930 2024-08-25T20:28:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.29c274ec4930 2024-08-25T20:28:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:28:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:28:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:28:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:28:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:28:02 2024 2024-08-25T20:28:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:28:02 2024 2024-08-25T20:28:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:28:02 2024 2024-08-25T20:28:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:28:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:28:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: e7e2ef0b-4966-416b-b3c4-b40984147b77 2024-08-25T20:28:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:28:03.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:28:03.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:28:03.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-08-25T20:28:03.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd0 2024-08-25T20:28:03.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:28:03.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-08-25T20:28:03.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:28:03.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:28:03.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:28:03.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 7fec4e56-63ee-418e-a2fb-6801a120df80 2024-08-25T20:28:03.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:28:03.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:28:03.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:03.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:28:03.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:28:03.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:28:03.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:28:03.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:03.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary1 2024-08-25T20:28:03.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-08-25T20:28:03.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:28:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary1 2024-08-25T20:28:03.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:28:03.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-08-25T20:28:03.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:03.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:28:03.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:03.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary2 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary1 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary1 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:03.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-08-25T20:28:03.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:28:03.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:28:03.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:28:03.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:03.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-08-25T20:28:03.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:28:03.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:03.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:28:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-08-25T20:28:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-08-25T20:28:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:28:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-08-25T20:28:04.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:28:04.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1654 2024-08-25T20:28:04.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:04.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-08-25T20:28:04.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary2': 2024-08-25T20:28:04.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:28:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:28:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:28:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 29d225c68b4b 2024-08-25T20:28:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.29d225c68b4b 2024-08-25T20:28:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:28:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:28:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:28:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:28:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:28:03 2024 2024-08-25T20:28:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:28:03 2024 2024-08-25T20:28:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:28:03 2024 2024-08-25T20:28:04.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:28:04.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:28:04.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: a369576b-ec7f-4cd4-8942-e9c93da688f5 2024-08-25T20:28:04.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:28:04.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:28:04.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:28:04.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-08-25T20:28:04.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd1 2024-08-25T20:28:04.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:28:04.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-08-25T20:28:04.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:28:04.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:28:04.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:28:04.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 6e3e64f9-65df-45fc-9f3e-27bed0151574 2024-08-25T20:28:04.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:28:04.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:28:04.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:04.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:28:04.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:28:04.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:28:04.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:28:04.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary2 2024-08-25T20:28:04.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-08-25T20:28:04.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:28:04.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary2 2024-08-25T20:28:04.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:28:04.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-08-25T20:28:04.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary3 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary2 2024-08-25T20:28:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:04.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:04.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:04.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:04.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:04.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:04.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:04.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:28:04.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:04.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-08-25T20:28:04.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary2 2024-08-25T20:28:04.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:04.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:04.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:04.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:04.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:04.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:04.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:28:04.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-08-25T20:28:04.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:28:04.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:28:04.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:04.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-08-25T20:28:04.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:28:04.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:04.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:28:04.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:04.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-08-25T20:28:04.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:04.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:04.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:04.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:04.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-08-25T20:28:05.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:28:05.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-08-25T20:28:05.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:28:05.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1656 2024-08-25T20:28:05.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-08-25T20:28:05.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:05.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary3': 2024-08-25T20:28:05.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:28:05.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:28:05.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:28:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 29e11f025d31 2024-08-25T20:28:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.29e11f025d31 2024-08-25T20:28:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:28:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:28:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:28:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:28:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:28:04 2024 2024-08-25T20:28:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:28:04 2024 2024-08-25T20:28:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:28:04 2024 2024-08-25T20:28:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:28:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:28:05.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 9b637205-cc57-43e2-ab62-6a8b2b9c7b2d 2024-08-25T20:28:05.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:28:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:28:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:28:05.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-08-25T20:28:05.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd2 2024-08-25T20:28:05.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:28:05.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-08-25T20:28:05.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:28:05.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:28:05.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:28:05.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 8350e244-4aa2-467e-b992-7f8910d5a2c3 2024-08-25T20:28:05.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:28:05.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:28:05.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:05.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:28:05.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:28:05.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:28:05.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:28:05.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:05.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary3 2024-08-25T20:28:05.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-08-25T20:28:05.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:28:05.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary3 2024-08-25T20:28:05.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:28:05.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-08-25T20:28:05.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary3 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary4 2024-08-25T20:28:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary3 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:05.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:05.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:05.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:05.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:05.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:28:05.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:28:05.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:28:05.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:05.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-08-25T20:28:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:28:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:28:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-08-25T20:28:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-08-25T20:28:06.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:28:06.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-08-25T20:28:06.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1658 2024-08-25T20:28:06.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:28:06.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:06.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-08-25T20:28:06.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary4': 2024-08-25T20:28:06.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:28:06.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:28:06.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:28:06.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 29f0729e8bbe 2024-08-25T20:28:06.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.29f0729e8bbe 2024-08-25T20:28:06.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:28:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:28:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:28:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:28:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:28:05 2024 2024-08-25T20:28:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:28:05 2024 2024-08-25T20:28:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:28:05 2024 2024-08-25T20:28:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:28:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:28:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: b603d324-2f1a-4cd8-a553-2dc40e9b9994 2024-08-25T20:28:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:28:06.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:28:06.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:28:06.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-08-25T20:28:06.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd3 2024-08-25T20:28:06.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:28:06.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-08-25T20:28:06.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:28:06.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:28:06.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:28:06.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: f2fcc434-683c-479a-84af-761ecdd8c67b 2024-08-25T20:28:06.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:28:06.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:28:06.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:06.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:28:06.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:28:06.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:28:06.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:28:06.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:06.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary4 2024-08-25T20:28:06.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-08-25T20:28:06.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:28:06.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary4 2024-08-25T20:28:06.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:28:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-08-25T20:28:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:28:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:28:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:28:06.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:28:06.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:28:06.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary5 2024-08-25T20:28:06.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-08-25T20:28:06.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary4 2024-08-25T20:28:06.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:06.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:06.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:06.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:06.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary4 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:28:06.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:28:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-08-25T20:28:06.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-08-25T20:28:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:28:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:28:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-08-25T20:28:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:06.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-08-25T20:28:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:28:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-08-25T20:28:07.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:07.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:07.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:07.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:07.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:07.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:07.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:28:07.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1660 2024-08-25T20:28:07.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-08-25T20:28:07.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary5': 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 29ff336eac42 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.29ff336eac42 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:28:06 2024 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:28:06 2024 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:28:06 2024 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:28:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 27def1ab-cb8e-459d-9676-6dc5bb3a2428 2024-08-25T20:28:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:28:07.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:28:07.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:28:07.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-08-25T20:28:07.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd4 2024-08-25T20:28:07.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:28:07.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-08-25T20:28:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:28:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:28:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:28:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: f9567a3c-f2bc-4977-bf74-021ec4923c27 2024-08-25T20:28:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:28:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:28:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:28:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:28:07.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:28:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:28:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:07.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary5 2024-08-25T20:28:07.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-08-25T20:28:07.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:28:07.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary5 2024-08-25T20:28:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:28:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-08-25T20:28:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:28:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:28:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary5 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary6 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary5 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:08.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-08-25T20:28:08.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:28:08.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:08.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:28:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-08-25T20:28:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:08.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-08-25T20:28:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:28:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-08-25T20:28:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:08.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1663 2024-08-25T20:28:08.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:28:08.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:08.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1662 2024-08-25T20:28:08.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-08-25T20:28:08.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:08.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary6': 2024-08-25T20:28:08.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:28:08.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:28:08.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:28:08.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2a106776c7c6 2024-08-25T20:28:08.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2a106776c7c6 2024-08-25T20:28:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:28:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:28:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:28:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:28:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:28:08 2024 2024-08-25T20:28:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:28:08 2024 2024-08-25T20:28:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:28:08 2024 2024-08-25T20:28:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:28:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:28:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 6c9ee9a6-a0b5-4565-947e-12f148eca87f 2024-08-25T20:28:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:28:08.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:28:08.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:28:08.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-08-25T20:28:08.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd5 2024-08-25T20:28:08.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:28:08.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-08-25T20:28:08.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:28:08.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:28:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:28:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 2b593866-2a5e-4af2-840c-56d5e054d4e1 2024-08-25T20:28:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:28:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:28:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:28:08.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:28:08.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:28:08.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:28:08.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:08.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary6 2024-08-25T20:28:08.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-08-25T20:28:09.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:28:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary6 2024-08-25T20:28:09.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:28:09.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary6 2024-08-25T20:28:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary7 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary6 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-08-25T20:28:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:28:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:28:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:28:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-08-25T20:28:09.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:28:09.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:09.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:28:09.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:09.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-08-25T20:28:09.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:09.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:09.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:09.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:09.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-08-25T20:28:09.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:28:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:28:09.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:28:09.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-08-25T20:28:09.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:28:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1666 2024-08-25T20:28:09.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1665 2024-08-25T20:28:09.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:09.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-08-25T20:28:09.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:09.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary7': 2024-08-25T20:28:09.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:28:09.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:28:09.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:28:09.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2a21b24badb3 2024-08-25T20:28:09.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2a21b24badb3 2024-08-25T20:28:09.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:28:09.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:28:09.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:28:09.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:28:09.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:28:09 2024 2024-08-25T20:28:09.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:28:09 2024 2024-08-25T20:28:09.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:28:09 2024 2024-08-25T20:28:09.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:28:09.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:28:09.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 31a35556-0c5d-41e5-9ae3-cd9b163ef396 2024-08-25T20:28:09.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:28:09.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:28:09.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:28:09.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-08-25T20:28:09.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd6 2024-08-25T20:28:09.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:28:09.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-08-25T20:28:09.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:28:09.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:28:09.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:28:09.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 5cfa942a-2a1e-412f-8b0b-98c02b4144e4 2024-08-25T20:28:09.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:28:09.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:28:09.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:09.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:28:09.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:28:09.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:28:09.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:28:09.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:09.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary7 2024-08-25T20:28:09.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-08-25T20:28:09.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:28:09.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary7 2024-08-25T20:28:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:28:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-08-25T20:28:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary8 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary7 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary7 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:28:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-08-25T20:28:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:28:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:28:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-08-25T20:28:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:28:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:28:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-08-25T20:28:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-08-25T20:28:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:28:10.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-08-25T20:28:10.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:28:10.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1668 2024-08-25T20:28:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1669 2024-08-25T20:28:10.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-08-25T20:28:10.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:10.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:10.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:28:10.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:10.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:10.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:10.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:28:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary8': 2024-08-25T20:28:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:28:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:28:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:28:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2a31ad0e0fbe 2024-08-25T20:28:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2a31ad0e0fbe 2024-08-25T20:28:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:28:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:28:10.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:28:10.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:28:10.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:28:10 2024 2024-08-25T20:28:10.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:28:10 2024 2024-08-25T20:28:10.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:28:10 2024 2024-08-25T20:28:10.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:28:10.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:28:10.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: e5f22a6f-8b4b-42b8-b3e5-b053140e92e2 2024-08-25T20:28:10.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:28:10.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:28:10.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:28:10.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-08-25T20:28:10.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd7 2024-08-25T20:28:10.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-08-25T20:28:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:28:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:28:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:28:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: c440c64e-73e7-418a-97e3-28ab390c1474 2024-08-25T20:28:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:28:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:28:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:28:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:28:10.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:28:10.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:28:10.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:10.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary8 2024-08-25T20:28:10.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-08-25T20:28:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:28:10.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary8 2024-08-25T20:28:10.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:28:10.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-08-25T20:28:10.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:10.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:10.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:10.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:10.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:28:10.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary9 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary8 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary8 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:28:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:10.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-08-25T20:28:10.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-08-25T20:28:10.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:28:10.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:10.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:28:10.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:10.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-08-25T20:28:10.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:10.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:10.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:10.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:10.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-08-25T20:28:11.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:28:11.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-08-25T20:28:11.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1672 2024-08-25T20:28:11.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:28:11.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1671 2024-08-25T20:28:11.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:11.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-08-25T20:28:11.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:11.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:28:11.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:11.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:11.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:11.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:28:11.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary9': 2024-08-25T20:28:11.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:28:11.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:28:11.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:28:11.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2a4319cef5f8 2024-08-25T20:28:11.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2a4319cef5f8 2024-08-25T20:28:11.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:28:11.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:28:11.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:28:11.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:28:11.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:28:10 2024 2024-08-25T20:28:11.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:28:10 2024 2024-08-25T20:28:11.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:28:10 2024 2024-08-25T20:28:11.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:28:11.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:28:11.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: b109da61-be86-4ef4-a8a7-95d5ee0e539e 2024-08-25T20:28:11.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:28:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:28:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:28:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-08-25T20:28:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:11.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:11.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:11.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd8 2024-08-25T20:28:11.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:28:11.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-08-25T20:28:11.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:28:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:28:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:28:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 6f0cb929-b490-4955-9416-89c965f0edfe 2024-08-25T20:28:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:28:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:28:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:28:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:28:11.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:28:11.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:28:11.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:11.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary9 2024-08-25T20:28:11.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-08-25T20:28:11.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:28:11.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary9 2024-08-25T20:28:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:28:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-08-25T20:28:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:11.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary9 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary10 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary9 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:11.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:28:11.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:28:11.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:28:11.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:28:11.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-08-25T20:28:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:28:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:28:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:28:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-08-25T20:28:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:28:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-08-25T20:28:12.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:28:12.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-08-25T20:28:12.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1675 2024-08-25T20:28:12.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:28:12.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1676 2024-08-25T20:28:12.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1674 2024-08-25T20:28:12.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-08-25T20:28:12.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:12.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:12.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:12.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:12.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:12.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:12.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:12.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:12.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:28:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:28:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary10': 2024-08-25T20:28:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:28:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:28:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:28:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2a57dddd89a1 2024-08-25T20:28:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2a57dddd89a1 2024-08-25T20:28:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:28:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:28:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:28:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:28:12.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:28:11 2024 2024-08-25T20:28:12.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:28:11 2024 2024-08-25T20:28:12.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:28:11 2024 2024-08-25T20:28:12.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:28:12.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:28:12.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 371a44ff-21a7-4f10-bb6e-04ba829deb66 2024-08-25T20:28:12.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:28:12.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:28:12.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:28:12.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-08-25T20:28:12.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd9 2024-08-25T20:28:12.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:28:12.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-08-25T20:28:12.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:28:12.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:28:12.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:28:12.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 9c7f336e-5d50-4352-a81a-ec9af12fcec5 2024-08-25T20:28:12.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:28:12.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:28:12.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:12.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:28:12.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:28:12.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:28:12.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:28:12.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:28:12.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary10 2024-08-25T20:28:12.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-08-25T20:28:12.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:28:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary10 2024-08-25T20:28:12.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:28:12.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-08-25T20:28:12.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:12.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:12.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:12.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:12.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:28:12.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:12.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:28:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:28:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:28:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary10 2024-08-25T20:28:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 82939 2024-08-25T20:28:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary10 2024-08-25T20:28:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-08-25T20:28:13.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:28:13.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-08-25T20:28:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1678 2024-08-25T20:28:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:28:13.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:13.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:13.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:13.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:28:13.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:13.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:28:13.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:13.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:13.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:13.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:28:13.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1680 2024-08-25T20:28:13.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1679 2024-08-25T20:28:13.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:13.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:13.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:14.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:14.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:28:14.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:14.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:14.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:14.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:28:14.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:14.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:28:14.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:14.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:14.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:14.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:28:14.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1681 2024-08-25T20:28:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1682 2024-08-25T20:28:14.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:14.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:15.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:15.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:15.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:15.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:15.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:15.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:15.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1683 2024-08-25T20:28:15.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:15.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:28:15.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:15.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:15.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:15.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:28:15.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1684 2024-08-25T20:28:15.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:15.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:28:15.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:15.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:15.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:15.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:28:15.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:15.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1686 2024-08-25T20:28:16.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1687 2024-08-25T20:28:16.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:16.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:16.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:16.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:16.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:16.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:16.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:16.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:16.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:16.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:28:16.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:16.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:16.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:16.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:28:16.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1685 2024-08-25T20:28:16.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:16.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:16.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:28:16.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:16.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:16.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:16.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:28:17.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:17.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:28:17.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:17.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:17.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:17.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:28:17.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:17.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:28:17.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:17.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:17.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:17.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:28:17.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1690 2024-08-25T20:28:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1688 2024-08-25T20:28:17.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1689 2024-08-25T20:28:17.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:17.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:17.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:18.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:18.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:28:18.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:18.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:18.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:18.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:28:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:28:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:18.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:18.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:28:18.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1691 2024-08-25T20:28:18.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1692 2024-08-25T20:28:18.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:18.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:19.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:19.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:28:19.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:19.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:19.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:19.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:28:19.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:19.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:28:19.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:19.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:19.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:19.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:28:19.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1694 2024-08-25T20:28:19.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1693 2024-08-25T20:28:19.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:19.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:19.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:19.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:19.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:19.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:19.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:19.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:20.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1696 2024-08-25T20:28:20.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1695 2024-08-25T20:28:20.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1697 2024-08-25T20:28:20.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:20.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:20.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:20.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:20.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:28:20.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:20.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:20.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:20.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:28:20.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:20.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:28:20.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:20.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:20.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:20.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:28:20.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:20.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:20.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:20.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:20.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:20.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:21.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1700 2024-08-25T20:28:21.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1699 2024-08-25T20:28:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1698 2024-08-25T20:28:21.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:21.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:21.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:21.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:21.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:28:21.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:21.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:21.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:21.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:28:21.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:21.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:28:21.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:21.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:21.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:21.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:28:22.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1701 2024-08-25T20:28:22.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1702 2024-08-25T20:28:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:28:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:28:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:22.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:22.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:22.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:28:22.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:22.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:22.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:22.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:28:23.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:23.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:28:23.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:23.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:23.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:23.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:28:23.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:23.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:28:23.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:23.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:23.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:23.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:28:23.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:23.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:23.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:23.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:23.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:23.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:23.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1704 2024-08-25T20:28:23.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1703 2024-08-25T20:28:23.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:23.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:24.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:24.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:24.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:28:24.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:28:24.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:24.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:28:24.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:24.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:24.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:24.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:28:24.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1707 2024-08-25T20:28:24.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:24.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1706 2024-08-25T20:28:24.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:25.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:25.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:28:25.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:25.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:25.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:25.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:28:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:28:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:28:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1709 2024-08-25T20:28:25.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1710 2024-08-25T20:28:25.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:25.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:26.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1705 2024-08-25T20:28:26.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:26.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1712 2024-08-25T20:28:26.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:26.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:26.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:28:26.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:26.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:26.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:26.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:28:26.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:26.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:28:26.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:26.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:26.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:26.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:28:27.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1708 2024-08-25T20:28:27.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:27.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:27.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:28:27.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:27.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:27.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:27.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:28:27.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:27.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:28:27.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:27.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:27.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:27.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:28:27.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1713 2024-08-25T20:28:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:28.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1711 2024-08-25T20:28:28.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:28.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1716 2024-08-25T20:28:28.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:28.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:28.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:28:28.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:28.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:28.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:28.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:28:28.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:28.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:28:28.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:28.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:28.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:28.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:28:29.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:29.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:29.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:29.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:29.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:29.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:29.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1714 2024-08-25T20:28:29.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:29.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:28:29.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:29.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:29.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:29.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:28:29.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:29.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1717 2024-08-25T20:28:29.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:30.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1715 2024-08-25T20:28:30.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:30.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:30.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:30.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:30.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:30.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:30.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1719 2024-08-25T20:28:30.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:30.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:30.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:28:30.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:30.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:30.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:30.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:28:31.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1718 2024-08-25T20:28:31.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:31.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1721 2024-08-25T20:28:31.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:31.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:31.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:28:31.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:31.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:31.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:31.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:28:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:28:31.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:31.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:31.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:31.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:28:32.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1720 2024-08-25T20:28:32.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:32.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1723 2024-08-25T20:28:32.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:28:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:28:32.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:32.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:28:32.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:32.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:32.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:32.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:28:33.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1722 2024-08-25T20:28:33.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:33.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1725 2024-08-25T20:28:33.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:33.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:33.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:28:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:28:33.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:33.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:33.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:33.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:33.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:33.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:34.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1724 2024-08-25T20:28:34.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:34.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:34.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:28:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:28:34.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:34.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:34.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:34.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:34.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:34.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1727 2024-08-25T20:28:34.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:35.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1726 2024-08-25T20:28:35.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:35.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:28:35.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:35.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:35.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:35.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:28:35.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:35.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:35.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:28:35.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:35.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:35.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:35.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:28:35.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1728 2024-08-25T20:28:35.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:35.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1729 2024-08-25T20:28:35.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:36.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1730 2024-08-25T20:28:36.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:36.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:28:36.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:36.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:36.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:36.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:28:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:28:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:28:36.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1731 2024-08-25T20:28:36.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:28:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:28:37.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1733 2024-08-25T20:28:37.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:37.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:37.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:28:37.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:37.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:37.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:37.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:28:38.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1732 2024-08-25T20:28:38.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:38.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:38.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:28:38.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:38.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:38.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:38.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:28:38.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1735 2024-08-25T20:28:38.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:38.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:38.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:38.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:38.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:38.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:38.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:38.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:38.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:28:38.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:38.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:38.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:38.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:28:39.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1734 2024-08-25T20:28:39.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:39.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:39.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:28:39.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:39.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:39.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:39.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:28:39.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1737 2024-08-25T20:28:39.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:40.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1736 2024-08-25T20:28:40.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1739 2024-08-25T20:28:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:40.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:40.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:28:40.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:40.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:40.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:40.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:28:40.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:41.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1738 2024-08-25T20:28:41.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:41.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1740 2024-08-25T20:28:41.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1741 2024-08-25T20:28:41.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:41.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:28:41.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:41.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:41.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:41.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:28:41.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:41.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:41.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:41.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:28:41.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:41.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:41.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:41.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:28:42.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:42.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:28:42.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:42.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:42.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:42.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:28:42.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1743 2024-08-25T20:28:42.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:42.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:42.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:28:42.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:42.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:42.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:42.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:28:43.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1742 2024-08-25T20:28:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1744 2024-08-25T20:28:43.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:43.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:28:43.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:43.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:43.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:43.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:28:43.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:43.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:43.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:43.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:28:43.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:43.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:43.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:43.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:28:43.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1745 2024-08-25T20:28:43.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:44.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:44.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:44.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:28:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:44.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1746 2024-08-25T20:28:44.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1748 2024-08-25T20:28:44.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:28:45.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:45.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:45.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:45.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:28:45.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1750 2024-08-25T20:28:45.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1747 2024-08-25T20:28:46.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:46.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:28:46.471+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae5004000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:28:46.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:46.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:28:46.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:46.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:46.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:46.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:28:46.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:46.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:28:46.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:46.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:46.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:46.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:28:46.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:46.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:28:46.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:46.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:46.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:46.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:28:47.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1749 2024-08-25T20:28:47.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1751 2024-08-25T20:28:47.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:47.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:47.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1753 2024-08-25T20:28:47.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:47.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:28:47.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:47.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:47.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:47.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:28:47.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:47.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:47.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:28:47.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:47.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:47.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:47.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:28:48.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1752 2024-08-25T20:28:48.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:48.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:48.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:48.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:48.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:48.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:48.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:49.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:49.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:28:49.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:49.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:49.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:49.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:28:49.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1754 2024-08-25T20:28:49.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1755 2024-08-25T20:28:49.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:49.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1757 2024-08-25T20:28:50.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1756 2024-08-25T20:28:50.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:50.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:28:50.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:50.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:50.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:50.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:28:50.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:50.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:28:50.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1759 2024-08-25T20:28:50.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:51.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1758 2024-08-25T20:28:51.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:51.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1761 2024-08-25T20:28:51.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1760 2024-08-25T20:28:51.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:51.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:51.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:28:51.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:51.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:51.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:51.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:28:52.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1763 2024-08-25T20:28:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:28:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:28:52.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:52.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:28:52.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:52.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:52.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:52.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:28:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:53.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1762 2024-08-25T20:28:53.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:53.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:53.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:28:53.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:53.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:53.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:53.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:28:53.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:53.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:28:53.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:53.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:53.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:53.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:28:53.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:53.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:28:53.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:53.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:53.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:53.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:28:53.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1764 2024-08-25T20:28:53.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:53.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1765 2024-08-25T20:28:53.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:54.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:54.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:54.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:54.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:54.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:54.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:54.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:54.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:28:54.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:54.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:54.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:28:54.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:28:54.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:54.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:54.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:54.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:54.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:54.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:54.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1767 2024-08-25T20:28:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:54.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1768 2024-08-25T20:28:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:28:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:28:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:28:55.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1769 2024-08-25T20:28:55.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1770 2024-08-25T20:28:55.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:55.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:56.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1766 2024-08-25T20:28:56.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:28:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:28:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:28:56.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:56.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:28:56.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:56.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:56.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:28:56.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:28:56.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:56.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:28:56.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:56.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:56.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:28:56.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:28:57.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1771 2024-08-25T20:28:57.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:57.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1773 2024-08-25T20:28:57.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:28:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:28:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:28:57.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:57.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:28:57.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:57.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:57.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:28:57.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:28:58.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1772 2024-08-25T20:28:58.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:28:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:28:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:28:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:28:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:28:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:28:59.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1774 2024-08-25T20:28:59.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1775 2024-08-25T20:28:59.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:28:59.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:28:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:28:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:28:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:28:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:28:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:59.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:28:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1779 2024-08-25T20:28:59.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:00.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1777 2024-08-25T20:29:00.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1776 2024-08-25T20:29:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:00.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:00.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:29:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:29:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:29:00.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1780 2024-08-25T20:29:00.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:01.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1778 2024-08-25T20:29:01.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:01.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:01.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:29:01.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:01.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:01.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:29:01.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:29:01.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1781 2024-08-25T20:29:01.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:02.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:02.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:29:02.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:02.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:02.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:29:02.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:29:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:29:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:29:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:29:02.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1783 2024-08-25T20:29:02.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:02.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:29:02.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:02.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:02.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:29:02.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:29:02.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:03.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1782 2024-08-25T20:29:03.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:03.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:29:03.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:03.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:03.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:29:03.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:29:03.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:03.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:29:03.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:03.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:03.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:29:03.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:29:03.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:29:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:29:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:29:03.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1785 2024-08-25T20:29:03.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:03.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1784 2024-08-25T20:29:03.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:04.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:04.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:29:04.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:04.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:04.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:29:04.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:29:04.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1788 2024-08-25T20:29:04.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1787 2024-08-25T20:29:04.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:04.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:04.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:04.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:04.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:04.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:04.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:04.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:05.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1790 2024-08-25T20:29:05.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:05.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:29:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:29:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:29:06.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1789 2024-08-25T20:29:06.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1786 2024-08-25T20:29:06.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:06.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:06.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:29:06.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:06.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:06.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:29:06.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:29:06.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:06.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:29:06.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:06.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:06.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:29:06.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:29:06.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:06.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:29:06.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:06.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:06.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:29:06.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:29:07.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1791 2024-08-25T20:29:07.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:07.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1793 2024-08-25T20:29:07.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:29:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:29:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:29:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:29:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:29:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:29:08.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1792 2024-08-25T20:29:08.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1794 2024-08-25T20:29:08.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:08.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:08.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:29:08.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:08.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:08.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:29:08.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:29:08.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:09.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1795 2024-08-25T20:29:09.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:09.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:29:09.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:09.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:09.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:29:09.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:29:09.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:09.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:29:09.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:09.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:09.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:29:09.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:29:09.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:09.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1797 2024-08-25T20:29:09.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1796 2024-08-25T20:29:09.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:09.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:10.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:10.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:10.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:10.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:10.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:10.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:10.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1799 2024-08-25T20:29:10.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:29:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:29:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:29:11.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1798 2024-08-25T20:29:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:11.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1801 2024-08-25T20:29:11.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:11.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:11.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:29:11.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:11.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:11.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:29:11.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:29:11.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:11.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:29:11.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:11.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:11.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:29:11.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:29:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1800 2024-08-25T20:29:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:29:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:29:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:29:12.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1803 2024-08-25T20:29:12.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:12.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:12.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:29:12.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:12.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:12.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:29:12.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:29:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:29:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:29:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:29:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1802 2024-08-25T20:29:13.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:13.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:13.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:29:13.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:13.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:13.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:29:13.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:29:13.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1805 2024-08-25T20:29:13.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:14.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:14.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:29:14.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:14.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:14.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:29:14.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:29:14.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:14.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:14.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:14.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:14.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:14.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:14.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1804 2024-08-25T20:29:14.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:15.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1807 2024-08-25T20:29:15.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:15.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:29:15.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:15.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:15.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:29:15.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:29:15.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1806 2024-08-25T20:29:15.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:29:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:29:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:29:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1809 2024-08-25T20:29:15.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:16.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1808 2024-08-25T20:29:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:16.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1811 2024-08-25T20:29:16.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:16.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:16.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:29:16.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:16.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:16.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:29:16.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:29:16.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:16.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:29:16.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:16.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:16.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:29:16.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:29:17.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1810 2024-08-25T20:29:17.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:17.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:29:17.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:17.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:17.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:29:17.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:29:17.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:17.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1813 2024-08-25T20:29:17.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:18.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1812 2024-08-25T20:29:18.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:18.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:18.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:29:18.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:18.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:18.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:29:18.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:29:18.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:18.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:29:18.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:18.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:18.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:29:18.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:29:18.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1815 2024-08-25T20:29:18.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:18.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:29:18.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:29:19.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:19.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:29:19.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:19.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:19.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:29:19.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:29:19.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:19.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:29:19.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:19.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:19.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:29:19.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:29:19.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1814 2024-08-25T20:29:19.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:19.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1817 2024-08-25T20:29:19.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1816 2024-08-25T20:29:19.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:19.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:20.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:20.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:20.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:20.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:20.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:20.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:20.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1819 2024-08-25T20:29:20.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:29:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:29:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:29:21.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1818 2024-08-25T20:29:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:21.480 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:29:21.479+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae50eac00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:29:21.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1821 2024-08-25T20:29:21.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:21.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:21.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:29:21.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:21.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:21.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:29:21.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:29:21.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:21.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:29:21.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:21.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:21.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:29:21.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:29:22.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1820 2024-08-25T20:29:22.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1823 2024-08-25T20:29:22.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:22.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:22.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:29:22.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:22.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:22.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:29:22.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:29:23.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1822 2024-08-25T20:29:23.460 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:29:23.460+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf51d400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:29:23.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1825 2024-08-25T20:29:23.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:23.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:23.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:29:23.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:23.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:23.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:29:23.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:29:24.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1824 2024-08-25T20:29:24.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:24.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:29:24.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:24.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:24.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:29:24.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:29:24.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:24.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:24.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:25.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1826 2024-08-25T20:29:25.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:25.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:29:25.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:25.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:25.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:29:25.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:29:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1827 2024-08-25T20:29:25.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:25.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:29:25.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:25.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:25.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:29:25.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:29:25.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:25.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:25.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1829 2024-08-25T20:29:25.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:26.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1828 2024-08-25T20:29:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:29:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:26.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:29:26.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:29:26.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:26.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1831 2024-08-25T20:29:26.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:26.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:29:26.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:26.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:26.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:29:26.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:29:26.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:27.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1830 2024-08-25T20:29:27.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:27.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:29:27.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:27.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:27.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:29:27.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:29:27.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:27.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1833 2024-08-25T20:29:27.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:28.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1832 2024-08-25T20:29:28.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:28.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:28.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:29:28.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:28.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:28.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:29:28.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:29:28.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1835 2024-08-25T20:29:28.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:28.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:29:28.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:28.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:28.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:29:28.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:29:28.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:28.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:28.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:29:28.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:28.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:28.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:29:28.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:29:29.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1834 2024-08-25T20:29:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:29:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:29:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:29:29.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:29.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:29:29.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:29.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:29.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:29:29.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:29:29.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1837 2024-08-25T20:29:29.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1836 2024-08-25T20:29:29.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:29.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:30.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:30.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:30.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:30.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:30.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:30.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:30.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1839 2024-08-25T20:29:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:30.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:30.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:29:30.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:30.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:30.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:29:30.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:29:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1838 2024-08-25T20:29:31.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:31.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1840 2024-08-25T20:29:31.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:31.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1841 2024-08-25T20:29:31.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:31.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:29:31.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:31.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:31.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:29:31.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:29:31.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:31.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:31.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:29:31.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:31.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:31.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:29:31.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:29:32.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:32.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:29:32.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:32.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:32.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:29:32.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:29:32.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1843 2024-08-25T20:29:32.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:29:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:29:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:29:32.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:32.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:29:32.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:32.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:32.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:29:32.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:29:33.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1842 2024-08-25T20:29:33.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:33.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:33.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:29:33.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:33.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:33.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:29:33.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:29:33.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1845 2024-08-25T20:29:33.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:29:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:29:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:29:34.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:34.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:29:34.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:34.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:34.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:29:34.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:29:34.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1844 2024-08-25T20:29:34.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:34.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:34.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:34.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:34.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:34.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:34.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:35.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1847 2024-08-25T20:29:35.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1846 2024-08-25T20:29:35.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:35.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:35.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:35.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:29:35.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:35.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:35.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:29:35.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:29:35.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1849 2024-08-25T20:29:35.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:36.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1848 2024-08-25T20:29:36.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:36.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:29:36.485+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae3408c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:29:36.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:36.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:29:36.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:36.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:36.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:29:36.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:29:36.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:36.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:29:36.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:36.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:36.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:29:36.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:29:37.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1851 2024-08-25T20:29:37.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1850 2024-08-25T20:29:37.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:37.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:29:37.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:37.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:37.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:29:37.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:29:37.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:37.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:37.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1853 2024-08-25T20:29:37.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:38.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1852 2024-08-25T20:29:38.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:38.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:29:38.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:38.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:38.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:29:38.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:29:38.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:38.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:29:38.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:38.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:38.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:29:38.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:29:38.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1855 2024-08-25T20:29:38.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:38.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:38.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:29:38.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:38.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:38.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:29:38.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:29:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:29:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:29:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:29:39.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1854 2024-08-25T20:29:39.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:39.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1857 2024-08-25T20:29:39.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1856 2024-08-25T20:29:39.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:39.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:40.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1859 2024-08-25T20:29:40.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:40.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:40.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:40.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:40.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:40.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:40.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:40.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:29:40.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:40.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:40.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:29:40.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:29:40.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:40.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:40.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:29:40.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:40.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:40.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:29:40.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:29:41.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1858 2024-08-25T20:29:41.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:41.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1861 2024-08-25T20:29:41.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1860 2024-08-25T20:29:41.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:41.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:41.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:41.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:29:41.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:41.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:41.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:29:41.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:29:41.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:41.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:29:41.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:41.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:41.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:29:41.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:29:42.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1863 2024-08-25T20:29:42.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:29:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:42.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:42.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:29:42.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:29:42.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:42.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:29:42.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:42.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:42.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:29:42.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:29:42.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:42.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:29:42.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:42.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:42.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:29:42.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:29:43.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1862 2024-08-25T20:29:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:43.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1865 2024-08-25T20:29:43.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:29:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:29:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:29:44.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1864 2024-08-25T20:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:29:44.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:44.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:44.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:29:44.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:44.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:44.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:29:44.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:29:44.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:44.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:44.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:44.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:44.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:44.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:45.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1866 2024-08-25T20:29:45.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1867 2024-08-25T20:29:45.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:45.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:45.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:29:45.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:45.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:45.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:29:45.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:29:45.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:45.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1869 2024-08-25T20:29:45.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1870 2024-08-25T20:29:45.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:45.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:46.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1868 2024-08-25T20:29:46.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:46.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:29:46.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:46.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:46.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:29:46.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:29:46.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:46.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:46.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:29:46.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:46.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:46.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:29:46.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:29:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1871 2024-08-25T20:29:47.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:47.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:47.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:29:47.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:47.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:47.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:29:47.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:29:47.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1873 2024-08-25T20:29:47.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:48.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1872 2024-08-25T20:29:48.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:48.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:29:48.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:48.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:48.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:29:48.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:29:48.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:29:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:29:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:29:48.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:48.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1875 2024-08-25T20:29:48.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:48.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:29:48.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:48.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:48.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:29:48.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:29:48.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:48.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:48.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:48.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:48.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:48.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:48.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1874 2024-08-25T20:29:49.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:49.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:29:49.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:49.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:49.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:29:49.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:29:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:49.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1879 2024-08-25T20:29:49.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1877 2024-08-25T20:29:49.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1876 2024-08-25T20:29:49.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:49.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:49.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:50.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:50.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:29:50.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:50.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:50.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:29:50.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:29:50.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:50.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:29:50.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:50.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:50.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:29:50.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:29:50.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1880 2024-08-25T20:29:50.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:51.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1878 2024-08-25T20:29:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:51.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1881 2024-08-25T20:29:51.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:51.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:51.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:29:51.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:51.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:51.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:29:51.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:29:51.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:51.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:29:51.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:51.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:51.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:29:51.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:29:52.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:52.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:29:52.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:52.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:52.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:29:52.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:29:52.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1883 2024-08-25T20:29:52.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:52.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:52.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:52.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:52.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:29:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:29:52.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:29:52.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:52.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:29:52.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:52.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:52.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:29:52.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:29:53.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1882 2024-08-25T20:29:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:53.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:53.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:29:53.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:53.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:53.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:29:53.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:29:53.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1886 2024-08-25T20:29:53.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1885 2024-08-25T20:29:53.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:53.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:54.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:54.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:29:54.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:54.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:54.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:29:54.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:29:54.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1884 2024-08-25T20:29:54.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:54.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:29:54.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:54.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:54.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:29:54.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:29:54.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:55.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1888 2024-08-25T20:29:55.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1887 2024-08-25T20:29:55.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:55.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:55.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:29:55.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:55.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:55.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:29:55.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:29:55.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:55.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1890 2024-08-25T20:29:55.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:56.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1889 2024-08-25T20:29:56.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:56.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:56.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:29:56.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:56.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:56.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:29:56.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:29:56.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:56.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:29:56.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:56.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:56.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:29:56.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:29:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:29:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:29:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:29:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1891 2024-08-25T20:29:57.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:57.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:57.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:29:57.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:57.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:57.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:29:57.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:29:57.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1893 2024-08-25T20:29:57.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:58.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1894 2024-08-25T20:29:58.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1892 2024-08-25T20:29:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:29:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:29:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:29:58.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:58.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:58.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:29:58.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:29:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:29:58.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:58.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1896 2024-08-25T20:29:58.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:58.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:58.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:29:58.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:58.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:58.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:29:58.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:29:59.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:29:59.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:29:59.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:29:59.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:29:59.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:29:59.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:29:59.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1895 2024-08-25T20:29:59.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1897 2024-08-25T20:29:59.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1898 2024-08-25T20:29:59.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:29:59.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:00.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:00.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:30:00.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:00.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:00.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:30:00.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:30:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1900 2024-08-25T20:30:00.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:30:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:00.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:30:00.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:30:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1899 2024-08-25T20:30:01.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1901 2024-08-25T20:30:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:01.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:01.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:30:01.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:01.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:01.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:30:01.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:30:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:30:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:30:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:30:01.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:01.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:30:01.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:01.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:01.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:30:01.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:30:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:30:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:30:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:30:02.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1903 2024-08-25T20:30:02.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:02.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:30:02.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:02.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:02.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:30:02.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:30:02.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:02.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:02.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:30:02.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:02.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:02.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:30:02.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:30:03.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1902 2024-08-25T20:30:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:30:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:30:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:30:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:03.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1906 2024-08-25T20:30:03.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:04.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:04.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:30:04.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:04.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:04.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:30:04.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:30:04.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:04.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:30:04.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:04.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:04.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:30:04.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:30:04.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1905 2024-08-25T20:30:04.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:05.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1908 2024-08-25T20:30:05.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1907 2024-08-25T20:30:05.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1904 2024-08-25T20:30:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:30:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:30:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:30:05.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:05.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:05.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1910 2024-08-25T20:30:05.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:06.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1909 2024-08-25T20:30:06.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:06.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:06.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:30:06.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:06.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:06.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:30:06.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:30:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:30:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:30:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:30:07.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1911 2024-08-25T20:30:07.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1913 2024-08-25T20:30:07.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:07.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:07.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:30:07.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:07.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:07.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:30:07.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:30:08.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1912 2024-08-25T20:30:08.486 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:30:08.485+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf84e000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:30:08.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:08.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1915 2024-08-25T20:30:08.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:08.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:08.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:30:08.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:08.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:30:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:30:08.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:08.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:30:08.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:08.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:08.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:30:08.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:30:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:30:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:30:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:30:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:30:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:08.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:30:08.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:30:09.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:09.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:30:09.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:09.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:09.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:30:09.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:30:09.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1914 2024-08-25T20:30:09.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:09.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1917 2024-08-25T20:30:09.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1918 2024-08-25T20:30:09.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:09.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:30:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:30:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:30:10.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:30:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:30:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:30:10.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1920 2024-08-25T20:30:10.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:11.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1919 2024-08-25T20:30:11.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:30:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:30:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:30:11.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1921 2024-08-25T20:30:11.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:30:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:30:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:30:12.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1916 2024-08-25T20:30:12.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1922 2024-08-25T20:30:12.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:12.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:12.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1923 2024-08-25T20:30:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:12.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:12.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:30:12.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:12.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:12.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:30:12.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:30:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:30:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:30:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:30:12.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:12.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:30:12.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:12.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:12.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:30:12.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:30:13.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1925 2024-08-25T20:30:13.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:13.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:13.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:30:13.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:13.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:13.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:30:13.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:30:14.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:14.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:30:14.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:14.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:14.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:30:14.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:30:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1924 2024-08-25T20:30:14.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:14.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:30:14.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:14.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:14.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:30:14.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:30:14.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:15.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:15.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:30:15.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:15.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:15.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:30:15.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:30:15.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:15.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:30:15.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:15.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:15.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:30:15.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:30:15.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:15.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:30:15.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:15.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:15.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:30:15.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:30:15.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1926 2024-08-25T20:30:15.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1927 2024-08-25T20:30:15.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:15.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:15.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1929 2024-08-25T20:30:15.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:16.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1928 2024-08-25T20:30:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:16.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1931 2024-08-25T20:30:16.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:16.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:30:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:30:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:30:17.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1930 2024-08-25T20:30:17.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:17.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1934 2024-08-25T20:30:17.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:17.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:17.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:30:17.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:17.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:17.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:30:17.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:30:18.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1932 2024-08-25T20:30:18.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1933 2024-08-25T20:30:18.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:18.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:18.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1935 2024-08-25T20:30:18.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:30:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:30:18.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:30:18.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:18.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:30:18.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:18.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:18.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:30:18.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:30:18.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:18.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:30:18.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:18.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:18.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:30:18.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:30:19.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:19.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:30:19.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:19.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:19.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:30:19.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:30:19.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:19.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:30:19.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:19.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:19.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:30:19.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:30:19.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1938 2024-08-25T20:30:19.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1937 2024-08-25T20:30:19.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:19.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:20.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:30:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:30:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:30:20.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:20.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:30:20.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:20.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:20.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:30:20.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:30:20.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1939 2024-08-25T20:30:20.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:21.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1936 2024-08-25T20:30:21.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:21.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:30:21.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:21.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:21.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:30:21.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:30:21.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:21.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:30:21.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:21.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:21.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:30:21.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:30:21.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:21.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:21.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:30:21.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:21.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:21.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:30:21.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:30:21.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1942 2024-08-25T20:30:21.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1941 2024-08-25T20:30:21.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:21.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:22.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1940 2024-08-25T20:30:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:22.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1944 2024-08-25T20:30:22.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1943 2024-08-25T20:30:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:22.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:30:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:30:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:30:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:30:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:30:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:30:23.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:23.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:30:23.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:23.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:23.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:30:23.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:30:24.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1945 2024-08-25T20:30:24.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:24.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:30:24.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:24.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:24.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:30:24.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:30:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:24.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:24.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:24.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:30:24.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:30:24.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:24.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:24.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:24.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:24.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:30:24.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:30:24.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:30:24.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:30:25.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1946 2024-08-25T20:30:25.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1947 2024-08-25T20:30:25.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:25.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:30:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:30:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:30:25.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:25.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:25.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1949 2024-08-25T20:30:25.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:25.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:30:25.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:25.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:25.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:30:25.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:30:25.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:25.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:30:25.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:25.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:25.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:30:25.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:30:25.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1950 2024-08-25T20:30:25.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:25.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:26.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1948 2024-08-25T20:30:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:26.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1952 2024-08-25T20:30:26.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:27.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1951 2024-08-25T20:30:27.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:27.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:30:27.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:27.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:27.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:30:27.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:30:27.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:28.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1953 2024-08-25T20:30:28.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1954 2024-08-25T20:30:28.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:28.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:28.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:28.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:30:28.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:28.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:28.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:30:28.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:30:28.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:28.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:30:28.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:28.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:28.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:30:28.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:30:28.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1955 2024-08-25T20:30:28.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:28.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:30:28.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83071 2024-08-25T20:30:28.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:28.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:30:28.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:28.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:28.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:30:28.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:30:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:30:29.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:29.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:29.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:30:29.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:30:29.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1956 2024-08-25T20:30:29.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1957 2024-08-25T20:30:29.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:29.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:30:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:30:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:30:30.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1958 2024-08-25T20:30:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:30.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:30:30.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83203 2024-08-25T20:30:30.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1959 2024-08-25T20:30:30.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:31.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:30:31.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83335 2024-08-25T20:30:31.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:31.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:30:31.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:31.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:31.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:30:31.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:30:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:30:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83487 2024-08-25T20:30:32.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1960 2024-08-25T20:30:32.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:32.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1961 2024-08-25T20:30:32.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:30:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:30:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:30:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:30:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:30:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:30:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:30:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83640 2024-08-25T20:30:33.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:30:33.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83793 2024-08-25T20:30:35.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1962 2024-08-25T20:30:35.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:30:35.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83946 2024-08-25T20:30:35.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:30:35.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:30:35.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 84119 2024-08-25T20:30:35.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:30:35.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 84293 2024-08-25T20:30:38.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:30:38.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:30:38.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 82940 2024-08-25T20:33:04.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:33:04.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:33:04.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:33:04.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83072 2024-08-25T20:33:05.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:33:05.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:33:05.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:33:05.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83204 2024-08-25T20:33:06.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:33:06.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:33:06.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:33:06.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83336 2024-08-25T20:33:07.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:33:07.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:33:07.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:33:07.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83488 2024-08-25T20:33:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:33:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:33:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:33:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83641 2024-08-25T20:33:09.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:33:09.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:33:09.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:33:09.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83794 2024-08-25T20:33:10.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:33:10.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:33:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:33:10.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 83947 2024-08-25T20:33:11.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:33:11.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:33:11.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:33:11.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 84120 2024-08-25T20:33:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:33:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:33:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:33:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 84294 2024-08-25T20:33:13.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:33:13.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:33:13.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:33:13.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:33:13.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 1 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd0 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary1 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary1 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary1 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 2 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd1 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary2 2024-08-25T20:33:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary2 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary2 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 3 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd2 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary3 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary3 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary3 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:33:13.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 4 2024-08-25T20:33:13.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd3 2024-08-25T20:33:13.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary4 2024-08-25T20:33:13.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary4 2024-08-25T20:33:13.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:33:13.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary4 2024-08-25T20:33:13.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:33:13.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:33:13.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 5 2024-08-25T20:33:13.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd4 2024-08-25T20:33:13.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary5 2024-08-25T20:33:13.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary5 2024-08-25T20:33:13.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:33:13.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary5 2024-08-25T20:33:13.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:33:13.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:33:13.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 6 2024-08-25T20:33:13.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd5 2024-08-25T20:33:13.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary6 2024-08-25T20:33:13.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary6 2024-08-25T20:33:13.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary6 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 7 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd6 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary7 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary7 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary7 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 8 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd7 2024-08-25T20:33:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:33:13.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary8 2024-08-25T20:33:13.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary8 2024-08-25T20:33:13.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:33:13.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary8 2024-08-25T20:33:13.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 9 2024-08-25T20:33:13.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:33:13.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:33:13.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd8 2024-08-25T20:33:13.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary9 2024-08-25T20:33:13.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary9 2024-08-25T20:33:13.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:33:13.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary9 2024-08-25T20:33:13.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:33:13.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 90252 2024-08-25T20:33:13.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 10 2024-08-25T20:33:13.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd9 2024-08-25T20:33:13.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary10 2024-08-25T20:33:13.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary10 2024-08-25T20:33:13.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:33:13.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary10 2024-08-25T20:33:13.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:33:13.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:33:13.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:33:13.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:33:13.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:33:13.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:33:14.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:33:14.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:33:14.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:33:14.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:33:14.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:33:14.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:33:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:33:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:33:14.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:33:14.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:33:14.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:33:14.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:33:14.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:33:14.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:34:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=607cbe92b000cdcf5feb4e24ab26e831 2024-08-25T20:34:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-08-25T20:34:03.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary5 2024-08-25T20:34:03.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:03.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:03.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:34:03.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-08-25T20:34:05.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:34:05.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-08-25T20:34:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:34:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:06.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:34:06.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:06.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:06.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:34:06.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:06.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:06.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:06.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:34:06.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=3e7923e107234861b25b0d03ec073e08 2024-08-25T20:34:06.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-08-25T20:34:06.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:34:06.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 27def1ab-cb8e-459d-9676-6dc5bb3a2428 2024-08-25T20:34:06.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:06.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:06.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:06.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:06.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:06.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:34:06.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary9 2024-08-25T20:34:06.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:06.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:06.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:34:06.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-08-25T20:34:07.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=2aa2221935dd3cae3c0373cb56535613 2024-08-25T20:34:07.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-08-25T20:34:07.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=1fb5d60417cecada98e09dc47715e30f 2024-08-25T20:34:07.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-08-25T20:34:07.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary6 2024-08-25T20:34:07.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:07.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:07.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:34:07.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-08-25T20:34:07.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary7 2024-08-25T20:34:07.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:07.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:07.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:34:07.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-08-25T20:34:07.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=ced08d822af9915d837c6d23353d070d 2024-08-25T20:34:07.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-08-25T20:34:08.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary2 2024-08-25T20:34:08.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:08.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:08.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:34:08.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-08-25T20:34:08.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:34:08.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:34:08.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-08-25T20:34:08.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:34:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:08.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-08-25T20:34:08.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:08.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:08.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:34:08.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:08.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:08.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:08.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:08.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:34:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:34:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:08.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:34:09.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=c61c5e0b398a48923f1ab3f321882580 2024-08-25T20:34:09.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-08-25T20:34:09.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:34:09.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 27def1ab-cb8e-459d-9676-6dc5bb3a2428 2024-08-25T20:34:09.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:09.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:09.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:09.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:09.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:09.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:09.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:09.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:34:09.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:34:09.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary8 2024-08-25T20:34:09.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:09.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:09.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:34:09.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-08-25T20:34:09.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-08-25T20:34:09.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:09.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:09.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:34:09.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:09.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:09.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:09.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:09.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:09.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:34:09.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:09.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:09.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:34:09.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:09.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:09.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:09.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:09.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:09.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:34:09.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:34:09.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:09.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:09.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:34:09.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:09.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:09.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:09.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:09.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:09.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:34:09.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=712c9cfeae825a605ea77e9dfef1a157 2024-08-25T20:34:09.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-08-25T20:34:09.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:34:09.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31a35556-0c5d-41e5-9ae3-cd9b163ef396 2024-08-25T20:34:09.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:09.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:09.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:09.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:09.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:09.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:09.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:09.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:34:09.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:34:09.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a369576b-ec7f-4cd4-8942-e9c93da688f5 2024-08-25T20:34:09.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:09.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:09.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:09.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:09.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:09.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:09.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:09.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:34:09.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary3 2024-08-25T20:34:09.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:09.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:09.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:34:09.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-08-25T20:34:10.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=a7dfef3699af18de8b57daa1e4bfe495 2024-08-25T20:34:10.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-08-25T20:34:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=9a55fae9aebe6adee79e93a5fdb5b142 2024-08-25T20:34:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-08-25T20:34:10.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary1 2024-08-25T20:34:10.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:10.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:10.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:34:10.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-08-25T20:34:10.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary4 2024-08-25T20:34:10.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:10.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:10.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:34:10.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-08-25T20:34:10.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:34:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-08-25T20:34:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:34:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:34:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:34:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:34:11.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:34:11.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6c9ee9a6-a0b5-4565-947e-12f148eca87f 2024-08-25T20:34:11.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:11.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:11.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:11.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:11.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:11.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:11.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:11.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:34:11.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:34:11.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:34:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:34:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:34:11.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-08-25T20:34:11.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:11.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:11.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:34:11.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:11.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:11.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:11.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:11.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:11.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-08-25T20:34:11.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:11.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:11.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:34:11.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:11.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:11.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:11.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:11.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-08-25T20:34:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:34:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-08-25T20:34:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:34:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:11.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=820c26778e8e5d5e3598b15e42f537e5 2024-08-25T20:34:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-08-25T20:34:11.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:34:11.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:11.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:11.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:34:11.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:11.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:11.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:11.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:11.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:11.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:34:11.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary10 2024-08-25T20:34:11.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:11.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:11.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:34:11.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-08-25T20:34:11.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:34:11.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9b637205-cc57-43e2-ab62-6a8b2b9c7b2d 2024-08-25T20:34:11.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:11.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:11.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:11.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:11.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:11.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:11.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:11.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:34:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:34:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:11.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:34:11.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:11.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:11.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:11.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:11.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:11.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:34:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:34:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:34:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:11.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:11.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:11.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:34:11.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:34:11.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31a35556-0c5d-41e5-9ae3-cd9b163ef396 2024-08-25T20:34:11.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:11.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:11.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:11.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:11.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:11.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:11.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:11.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:34:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:34:11.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a369576b-ec7f-4cd4-8942-e9c93da688f5 2024-08-25T20:34:11.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:11.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:11.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:11.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:11.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:11.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:11.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:11.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:34:12.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:34:12.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:12.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:12.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:34:12.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:12.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:12.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:12.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:12.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:12.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:34:12.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:34:12.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:12.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:12.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:34:12.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:12.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:12.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:34:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:34:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:34:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:34:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:34:12.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:12.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:12.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:12.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:12.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:12.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:34:12.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:12.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:34:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:34:12.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-08-25T20:34:12.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:12.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:12.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:34:12.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:12.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:12.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:12.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:12.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:12.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:34:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e5f22a6f-8b4b-42b8-b3e5-b053140e92e2 2024-08-25T20:34:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:34:12.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:34:12.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b109da61-be86-4ef4-a8a7-95d5ee0e539e 2024-08-25T20:34:12.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:12.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:12.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:12.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:12.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:34:12.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:34:12.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e7e2ef0b-4966-416b-b3c4-b40984147b77 2024-08-25T20:34:12.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:12.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:12.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:12.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:12.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:12.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:34:12.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b603d324-2f1a-4cd8-a553-2dc40e9b9994 2024-08-25T20:34:12.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:12.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:12.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:12.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:12.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:12.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:12.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:12.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:34:12.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:12.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:12.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:34:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:34:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:34:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:13.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:13.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:13.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:13.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:13.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:34:13.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:34:13.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6c9ee9a6-a0b5-4565-947e-12f148eca87f 2024-08-25T20:34:13.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:13.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:13.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:13.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:13.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:13.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:13.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:13.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:34:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:34:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:34:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:13.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:13.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:34:13.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:34:13.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 27def1ab-cb8e-459d-9676-6dc5bb3a2428 2024-08-25T20:34:13.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:13.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:13.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:13.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:13.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:13.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:13.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:13.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:13.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:34:13.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:13.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:13.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:34:13.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:13.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:34:13.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:34:13.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 371a44ff-21a7-4f10-bb6e-04ba829deb66 2024-08-25T20:34:13.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:13.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:13.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:13.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:13.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:13.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:13.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:13.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:34:13.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:34:13.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:13.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:13.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:34:13.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:13.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:13.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:13.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:13.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:13.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:34:13.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:34:13.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9b637205-cc57-43e2-ab62-6a8b2b9c7b2d 2024-08-25T20:34:13.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:13.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:13.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:13.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:13.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:34:14.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:34:14.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:14.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:14.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:34:14.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:14.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:14.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:14.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:14.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:14.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:34:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:34:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:34:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:14.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:14.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:34:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:34:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:34:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:34:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:34:14.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:14.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:14.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:14.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:14.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:14.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:34:14.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:14.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:34:14.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:34:14.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b109da61-be86-4ef4-a8a7-95d5ee0e539e 2024-08-25T20:34:14.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:14.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:14.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:14.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:14.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:34:14.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e5f22a6f-8b4b-42b8-b3e5-b053140e92e2 2024-08-25T20:34:14.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:14.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:14.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:14.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:14.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:14.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:14.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:14.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:14.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:34:14.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:14.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:14.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:34:14.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:34:14.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e7e2ef0b-4966-416b-b3c4-b40984147b77 2024-08-25T20:34:14.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:14.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:14.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:14.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:14.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:34:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b603d324-2f1a-4cd8-a553-2dc40e9b9994 2024-08-25T20:34:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:34:14.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:14.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:14.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:34:15.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:34:15.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:15.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:15.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:34:15.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:15.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:15.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:15.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:15.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:15.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:34:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:34:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 371a44ff-21a7-4f10-bb6e-04ba829deb66 2024-08-25T20:34:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:15.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:15.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:15.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:34:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:34:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:34:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:34:15.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:34:15.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:15.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:15.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:34:15.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:15.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:15.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:15.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:15.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:15.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:34:15.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:34:15.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31a35556-0c5d-41e5-9ae3-cd9b163ef396 2024-08-25T20:34:15.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:15.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:15.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:15.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:15.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:15.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:15.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:15.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:34:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a369576b-ec7f-4cd4-8942-e9c93da688f5 2024-08-25T20:34:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:15.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:15.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:15.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:34:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:34:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:17.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:17.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:34:17.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:34:17.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6c9ee9a6-a0b5-4565-947e-12f148eca87f 2024-08-25T20:34:17.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:17.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:17.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:17.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:17.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:17.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:17.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:17.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:17.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:34:17.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:17.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:17.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:34:17.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:17.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:17.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:17.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:17.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:17.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:34:17.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:34:17.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9b637205-cc57-43e2-ab62-6a8b2b9c7b2d 2024-08-25T20:34:17.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:17.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:17.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:17.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:17.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:17.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:17.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:17.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:18.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:34:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:34:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:34:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:34:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:34:18.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:34:18.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:18.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:18.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:34:18.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:18.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:18.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:18.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:18.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:18.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:34:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:34:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b109da61-be86-4ef4-a8a7-95d5ee0e539e 2024-08-25T20:34:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:34:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e5f22a6f-8b4b-42b8-b3e5-b053140e92e2 2024-08-25T20:34:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:34:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e7e2ef0b-4966-416b-b3c4-b40984147b77 2024-08-25T20:34:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:18.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:18.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:18.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:18.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:18.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:18.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:18.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:18.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:18.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:18.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:34:18.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b603d324-2f1a-4cd8-a553-2dc40e9b9994 2024-08-25T20:34:18.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:18.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:18.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:18.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:18.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:18.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:18.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:18.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:19.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:34:19.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:19.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:19.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:34:19.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:19.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:19.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:19.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:19.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:19.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:34:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:34:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 371a44ff-21a7-4f10-bb6e-04ba829deb66 2024-08-25T20:34:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:19.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:19.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:19.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:19.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:19.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:19.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:19.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:21.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:34:21.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:21.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:21.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:34:21.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:21.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:21.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:21.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:21.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:21.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:34:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:34:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 27def1ab-cb8e-459d-9676-6dc5bb3a2428 2024-08-25T20:34:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:21.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:21.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:21.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:21.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:23.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:34:23.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:23.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:23.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:34:23.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:23.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:23.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:23.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:23.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:23.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:34:23.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:34:23.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:23.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:23.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:34:23.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:23.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:23.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:23.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:23.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:23.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:34:24.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:34:24.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31a35556-0c5d-41e5-9ae3-cd9b163ef396 2024-08-25T20:34:24.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:24.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:24.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:24.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:24.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:24.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:24.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:24.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:24.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:34:24.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a369576b-ec7f-4cd4-8942-e9c93da688f5 2024-08-25T20:34:24.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:24.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:24.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:24.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:24.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:24.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:24.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:24.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:34:25.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:25.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:25.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:34:25.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:25.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:25.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:25.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:25.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:25.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:34:25.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:34:25.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6c9ee9a6-a0b5-4565-947e-12f148eca87f 2024-08-25T20:34:25.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:25.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:25.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:25.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:25.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:25.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:25.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:25.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:25.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:34:25.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:25.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:25.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:34:25.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:25.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:25.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:25.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:25.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:25.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:34:25.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:34:25.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9b637205-cc57-43e2-ab62-6a8b2b9c7b2d 2024-08-25T20:34:25.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:25.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:25.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:25.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:25.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:25.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:25.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:25.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:34:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:34:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:26.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:34:26.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:26.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:26.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:34:26.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:26.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:26.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:26.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:26.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:34:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:34:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:34:26.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:34:26.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:26.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:26.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:34:26.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:26.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:26.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:26.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:26.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:26.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:34:26.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:34:26.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b109da61-be86-4ef4-a8a7-95d5ee0e539e 2024-08-25T20:34:26.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:26.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:26.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:26.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:26.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:26.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:34:26.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e7e2ef0b-4966-416b-b3c4-b40984147b77 2024-08-25T20:34:26.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:26.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:26.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:26.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:26.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:26.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:34:26.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e5f22a6f-8b4b-42b8-b3e5-b053140e92e2 2024-08-25T20:34:26.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:26.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:26.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:26.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:26.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:26.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:26.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:26.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:26.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:26.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:26.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:26.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:34:26.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b603d324-2f1a-4cd8-a553-2dc40e9b9994 2024-08-25T20:34:26.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:26.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:26.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:26.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:26.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:26.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:26.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:26.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:26.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:26.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:26.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:27.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:34:27.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:27.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:27.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:34:27.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:27.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:27.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:27.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:27.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:27.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:34:27.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:34:27.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 371a44ff-21a7-4f10-bb6e-04ba829deb66 2024-08-25T20:34:27.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:34:27.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:34:27.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:27.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:33:58' 2024-08-25T20:34:27.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:27.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:34:27.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:27.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:34:29.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:34:29.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:29.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:29.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:34:29.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:29.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:29.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:29.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:29.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:29.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:34:29.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:34:29.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 27def1ab-cb8e-459d-9676-6dc5bb3a2428 2024-08-25T20:34:29.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:29.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:29.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:29.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:29.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:29.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:29.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:29.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:29.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:29.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:29.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:29.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:29.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:29.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:29.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:29.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:29.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:29.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:29.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:29.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:29.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:34:29.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:29.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:29.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:29.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:29.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:29.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:29.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:29.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:34:29.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:29.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:34:29.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:29.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:29.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-08-25T20:34:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:29.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:29.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:34:29.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:29.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:29.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:29.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:29.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:30.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-08-25T20:34:30.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:34:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-08-25T20:34:30.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:34:30.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 27def1ab-cb8e-459d-9676-6dc5bb3a2428 2024-08-25T20:34:30.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:30.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:30.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:30.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:30.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:30.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:30.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:30.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:30.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:30.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:30.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:30.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:30.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:30.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:30.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:30.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:34:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:31.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:31.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:31.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:34:31.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:31.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:34:31.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:31.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:31.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:31.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:31.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:31.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:31.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary5 2024-08-25T20:34:31.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:31.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:31.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:34:31.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:34:31.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-08-25T20:34:31.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:34:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:34:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:34:31.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-08-25T20:34:31.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd0 2024-08-25T20:34:31.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:34:31.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:34:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:34:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:34:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:32.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:32.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:34:32.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:34:32.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:32.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:32.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:34:32.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:32.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:32.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:32.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:32.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:32.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:34:32.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:34:32.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31a35556-0c5d-41e5-9ae3-cd9b163ef396 2024-08-25T20:34:32.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:32.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:32.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:32.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:32.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:32.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:32.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:32.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:32.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:32.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:32.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:32.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:32.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:32.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:32.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:32.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:32.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:32.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:32.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:32.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:32.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:34:32.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:32.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:32.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:34:32.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a369576b-ec7f-4cd4-8942-e9c93da688f5 2024-08-25T20:34:32.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:32.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:32.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:32.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:32.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:32.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:32.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:32.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:32.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:32.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:32.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:32.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:32.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:32.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:32.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:32.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:32.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:32.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:32.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:32.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:32.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:34:32.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:32.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:32.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:32.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:32.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:32.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:32.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:32.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:34:32.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:32.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:34:32.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:32.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:32.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:32.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:32.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:32.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:32.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-08-25T20:34:32.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:32.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:32.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:34:32.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:32.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:32.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:32.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:32.276 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:32.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:32.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:32.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:34:32.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:32.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:34:32.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:32.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:32.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:32.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-08-25T20:34:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:34:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:32.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:32.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-08-25T20:34:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:34:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:33.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:33.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:33.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-08-25T20:34:33.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:34:33.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:33.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:33.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:34:33.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:33.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:33.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:33.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:33.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:33.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:34:33.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-08-25T20:34:33.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:33.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:33.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:34:33.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:33.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:33.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:33.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:33.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:33.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-08-25T20:34:33.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:34:33.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31a35556-0c5d-41e5-9ae3-cd9b163ef396 2024-08-25T20:34:33.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:33.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:33.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:33.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:33.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:33.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:33.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:33.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:33.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:33.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:33.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:33.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:33.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:33.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:33.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:33.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:34:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:33.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:33.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:33.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:34:33.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6c9ee9a6-a0b5-4565-947e-12f148eca87f 2024-08-25T20:34:33.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:33.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:33.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:33.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:33.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:33.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:33.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:33.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:34:33.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a369576b-ec7f-4cd4-8942-e9c93da688f5 2024-08-25T20:34:33.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:33.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:33.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:33.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:33.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:33.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:33.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:33.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:33.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:33.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:33.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:33.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:33.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:34:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:33.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:33.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:33.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:33.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:33.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:33.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:33.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:34:33.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:33.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:33.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:33.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:33.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:33.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:33.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:33.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:34:33.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:33.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:34:33.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:33.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:33.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:33.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:33.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:33.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:33.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary7 2024-08-25T20:34:33.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:33.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:33.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:34:33.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:34:33.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-08-25T20:34:33.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:33.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:33.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:34:33.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:33.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:34:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:33.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:33.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:34:33.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:33.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:33.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:33.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:33.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:34:33.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:33.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:33.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:33.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-08-25T20:34:33.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:33.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:33.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:34:33.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:33.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:33.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:33.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:33.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:33.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:33.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:33.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:33.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:33.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:33.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:33.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary2 2024-08-25T20:34:33.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:33.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:33.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:34:33.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:34:33.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-08-25T20:34:33.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:34:33.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:33.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:33.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:34:33.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:33.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:33.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:33.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:33.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:33.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:34:34.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:34:34.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9b637205-cc57-43e2-ab62-6a8b2b9c7b2d 2024-08-25T20:34:34.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:34.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:34.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:34.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:34.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:34.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:34.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:34.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:34.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:34.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:34.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:34.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:34.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:34.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:34.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:34.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:34.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:34.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:34.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:34.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:34.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:34:34.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:34.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:34.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:34.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:34.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:34.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:34.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:34:34.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:34.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:34:34.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:34.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:34.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:34.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:34.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:34.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-08-25T20:34:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:34:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:34.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:34:34.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:34:34.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:34:34.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:34:34.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:34:34.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:34:34.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-08-25T20:34:34.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-08-25T20:34:34.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd2 2024-08-25T20:34:34.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd1 2024-08-25T20:34:34.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:34:34.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:34:34.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:34:34.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:34:34.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-08-25T20:34:34.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:34.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:34.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:34:34.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:34.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:34.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:34.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:34.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:34.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-08-25T20:34:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:34:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6c9ee9a6-a0b5-4565-947e-12f148eca87f 2024-08-25T20:34:34.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:34.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:34.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:34.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:34.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:34.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:34.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:34.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:34.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:34.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:34.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:34.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:34.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:34.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:34:34.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:34.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:34.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:34.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:34.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:34.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:34.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:34.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:34:34.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:34.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:34:34.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:34.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:34.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:34.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:34.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:34.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary6 2024-08-25T20:34:34.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:34.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:34.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:34:34.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:34:34.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-08-25T20:34:34.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:34:34.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:34.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:34.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:34:34.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:34.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:34.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:34.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:34.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:34:34.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:34.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:34.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:34:34.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:34.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:34.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:34.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:34.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:34.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:34:34.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:34:34.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:34.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:34.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:34:34.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:34:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:34:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:34:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:34.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:34.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:34:34.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:34.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:34.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:34.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:34.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:34.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:34:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:34:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e5f22a6f-8b4b-42b8-b3e5-b053140e92e2 2024-08-25T20:34:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:34.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:34.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:35.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:35.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:35.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:35.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:35.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:35.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:34:35.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b603d324-2f1a-4cd8-a553-2dc40e9b9994 2024-08-25T20:34:35.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:35.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:35.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:35.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:35.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:35.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:35.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:35.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:34:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b109da61-be86-4ef4-a8a7-95d5ee0e539e 2024-08-25T20:34:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:34:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e7e2ef0b-4966-416b-b3c4-b40984147b77 2024-08-25T20:34:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:35.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:35.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:35.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:35.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:35.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:35.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:35.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:35.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:35.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:34:35.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:35.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:35.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:35.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:35.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:35.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:35.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:35.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:35.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:35.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:35.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:35.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:35.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:35.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:35.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:35.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:35.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:35.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:35.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:35.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:35.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:35.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:34:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:35.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:35.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:35.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:35.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:35.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:35.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:34:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:34:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:35.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-08-25T20:34:35.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:35.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:35.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:34:35.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:35.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:35.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:35.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:35.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:35.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-08-25T20:34:35.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:35.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:34:35.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:35.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:35.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:34:35.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:34:35.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:35.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:34:35.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:35.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:35.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:35.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:35.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:35.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:35.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:35.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:35.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-08-25T20:34:35.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:35.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:35.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:34:35.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:35.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:35.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:35.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:35.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:35.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:35.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:35.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:35.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:35.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-08-25T20:34:35.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:35.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:35.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:34:35.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:35.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:35.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:35.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:35.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:35.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:34:35.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:35.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:35.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:34:35.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:35.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:34:35.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:35.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:34:35.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:35.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:35.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:35.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:35.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:35.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:35.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:35.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-08-25T20:34:35.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:35.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:35.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:34:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:35.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:35.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:35.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:35.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:35.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-08-25T20:34:35.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:35.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:35.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:34:35.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:35.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:35.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:35.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:35.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:35.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:34:35.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9b637205-cc57-43e2-ab62-6a8b2b9c7b2d 2024-08-25T20:34:35.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:35.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:35.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:35.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:35.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:35.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:35.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:35.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:35.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:35.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:35.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:35.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:35.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:35.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:35.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:35.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:35.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:35.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:35.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:35.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:35.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:34:35.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:35.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:35.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:35.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:35.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:35.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:34:35.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:34:35.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:34:35.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-08-25T20:34:35.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:35.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:35.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:34:35.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:35.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:34:35.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:35.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:35.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:35.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:35.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:35.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary3 2024-08-25T20:34:35.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:35.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:35.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:34:35.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:34:35.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-08-25T20:34:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd3 2024-08-25T20:34:35.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:34:35.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:34:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:34:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:34:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:35.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:35.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:35.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:34:36.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:34:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 371a44ff-21a7-4f10-bb6e-04ba829deb66 2024-08-25T20:34:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:36.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:36.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:36.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:36.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:36.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:36.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:36.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:36.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:36.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:34:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:34:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:36.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:36.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:34:36.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:36.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:34:36.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:36.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:36.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:36.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:36.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:36.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:36.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-08-25T20:34:36.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:36.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:34:36.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:36.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:36.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:36.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:34:36.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:34:36.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-08-25T20:34:36.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:36.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:34:36.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:36.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:36.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:36.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:36.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:36.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-08-25T20:34:36.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-08-25T20:34:36.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:36.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:34:36.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:36.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:36.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:36.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:36.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:36.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-08-25T20:34:36.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-08-25T20:34:36.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:36.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:34:36.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:36.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:36.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:36.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:36.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:36.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-08-25T20:34:36.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-08-25T20:34:36.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:36.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:34:36.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:36.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:36.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:36.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:36.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:36.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-08-25T20:34:36.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:34:36.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e7e2ef0b-4966-416b-b3c4-b40984147b77 2024-08-25T20:34:36.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:36.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:36.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:36.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:36.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:36.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:36.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:36.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:36.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:36.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:36.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:36.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:36.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:34:36.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b109da61-be86-4ef4-a8a7-95d5ee0e539e 2024-08-25T20:34:36.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:36.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:36.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:36.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:36.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:36.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:34:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e5f22a6f-8b4b-42b8-b3e5-b053140e92e2 2024-08-25T20:34:36.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:36.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:36.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:36.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:36.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:36.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:36.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:34:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:36.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:36.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:36.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:36.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:36.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:36.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:36.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:36.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:36.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:36.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:36.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:36.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:36.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:36.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:36.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:34:36.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:36.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:36.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:36.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:34:36.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b603d324-2f1a-4cd8-a553-2dc40e9b9994 2024-08-25T20:34:36.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:36.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:36.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:36.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:36.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:36.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:36.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:36.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:36.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:36.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:36.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:36.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:36.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:36.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:36.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:36.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:34:36.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:36.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:36.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:36.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:36.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:36.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:36.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:36.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:36.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:36.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:36.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:36.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:36.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:34:36.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:36.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:36.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:36.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:36.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:34:36.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:34:36.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-08-25T20:34:36.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:36.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:36.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:36.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:36.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:36.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:36.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:36.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:34:36.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:36.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:36.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:36.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:36.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:36.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:36.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:36.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:34:36.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:36.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:34:36.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:36.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:36.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:36.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:36.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:36.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:36.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary1 2024-08-25T20:34:36.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:36.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:34:36.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:34:36.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-08-25T20:34:36.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd4 2024-08-25T20:34:36.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:34:36.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:34:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:34:36.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:36.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:36.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:34:36.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:36.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:34:36.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:36.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:34:36.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:36.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:36.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:36.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:36.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:36.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:36.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:36.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary4 2024-08-25T20:34:36.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:36.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:34:36.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:34:36.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-08-25T20:34:36.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:34:36.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:36.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:34:36.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:36.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:36.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:36.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:36.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:36.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:36.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary9 2024-08-25T20:34:36.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:36.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:36.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:34:36.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:34:36.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-08-25T20:34:37.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-08-25T20:34:37.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:37.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:37.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:34:37.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:37.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:34:37.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:34:37.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:37.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:37.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-08-25T20:34:37.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:34:37.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 371a44ff-21a7-4f10-bb6e-04ba829deb66 2024-08-25T20:34:37.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:34:37.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:34:37.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:34:37.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:34:28' 2024-08-25T20:34:37.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:34:37.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:34:37.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:34:37.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:34:37.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:34:37.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:37.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:34:37.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:37.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:34:37.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:34:37.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:34:37.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:34:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-08-25 20:34:28' 2024-08-25T20:34:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:34:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:34:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:34:28' 2024-08-25T20:34:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:34:37.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:34:37.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:34:37.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:34:37.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:34:37.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:34:37.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:34:37.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:34:37.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:34:37.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-08-25T20:34:37.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:34:37.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:34:37.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-08-25T20:34:37.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:34:37.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:34:37.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:34:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:34:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-08-25T20:34:37.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-08-25T20:34:37.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:34:28' 2024-08-25T20:34:37.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:34:37.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:34:37.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:34:28' 2024-08-25T20:34:37.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:34:37.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:34:37.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:34:37.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:34:37.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:34:37.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:37.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:34:37.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary10 2024-08-25T20:34:37.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:34:37.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:34:37.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:34:37.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:34:37.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-08-25T20:34:37.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd5 2024-08-25T20:34:37.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:34:37.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:34:37.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd6 2024-08-25T20:34:37.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:34:37.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:34:37.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd7 2024-08-25T20:34:37.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:34:37.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:34:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd8 2024-08-25T20:34:38.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:34:38.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:34:38.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:34:38.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:34:38.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:34:38.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-08-25T20:34:38.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd9 2024-08-25T20:34:38.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:34:38.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:35:23.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=607cbe92b000cdcf5feb4e24ab26e831 2024-08-25T20:35:23.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-08-25T20:35:24.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 607cbe92b000cdcf5feb4e24ab26e831 != \6\0\7\c\b\e\9\2\b\0\0\0\c\d\c\f\5\f\e\b\4\e\2\4\a\b\2\6\e\8\3\1 ]] 2024-08-25T20:35:26.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=1fb5d60417cecada98e09dc47715e30f 2024-08-25T20:35:26.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-08-25T20:35:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 1fb5d60417cecada98e09dc47715e30f != \1\f\b\5\d\6\0\4\1\7\c\e\c\a\d\a\9\8\e\0\9\d\c\4\7\7\1\5\e\3\0\f ]] 2024-08-25T20:35:29.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=3e7923e107234861b25b0d03ec073e08 2024-08-25T20:35:29.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-08-25T20:35:29.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 3e7923e107234861b25b0d03ec073e08 != \3\e\7\9\2\3\e\1\0\7\2\3\4\8\6\1\b\2\5\b\0\d\0\3\e\c\0\7\3\e\0\8 ]] 2024-08-25T20:35:30.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=712c9cfeae825a605ea77e9dfef1a157 2024-08-25T20:35:30.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-08-25T20:35:30.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 712c9cfeae825a605ea77e9dfef1a157 != \7\1\2\c\9\c\f\e\a\e\8\2\5\a\6\0\5\e\a\7\7\e\9\d\f\e\f\1\a\1\5\7 ]] 2024-08-25T20:35:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=ced08d822af9915d837c6d23353d070d 2024-08-25T20:35:30.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-08-25T20:35:31.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ ced08d822af9915d837c6d23353d070d != \c\e\d\0\8\d\8\2\2\a\f\9\9\1\5\d\8\3\7\c\6\d\2\3\3\5\3\d\0\7\0\d ]] 2024-08-25T20:35:31.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=a7dfef3699af18de8b57daa1e4bfe495 2024-08-25T20:35:31.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-08-25T20:35:31.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=820c26778e8e5d5e3598b15e42f537e5 2024-08-25T20:35:31.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-08-25T20:35:31.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=2aa2221935dd3cae3c0373cb56535613 2024-08-25T20:35:31.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-08-25T20:35:31.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ a7dfef3699af18de8b57daa1e4bfe495 != \a\7\d\f\e\f\3\6\9\9\a\f\1\8\d\e\8\b\5\7\d\a\a\1\e\4\b\f\e\4\9\5 ]] 2024-08-25T20:35:31.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:35:31.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 90253 2024-08-25T20:35:31.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:35:31.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 90255 2024-08-25T20:35:31.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:35:31.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 90257 2024-08-25T20:35:31.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 820c26778e8e5d5e3598b15e42f537e5 != \8\2\0\c\2\6\7\7\8\e\8\e\5\d\5\e\3\5\9\8\b\1\5\e\4\2\f\5\3\7\e\5 ]] 2024-08-25T20:35:31.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 2aa2221935dd3cae3c0373cb56535613 != \2\a\a\2\2\2\1\9\3\5\d\d\3\c\a\e\3\c\0\3\7\3\c\b\5\6\5\3\5\6\1\3 ]] 2024-08-25T20:35:31.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=9a55fae9aebe6adee79e93a5fdb5b142 2024-08-25T20:35:31.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-08-25T20:35:31.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 9a55fae9aebe6adee79e93a5fdb5b142 != \9\a\5\5\f\a\e\9\a\e\b\e\6\a\d\e\e\7\9\e\9\3\a\5\f\d\b\5\b\1\4\2 ]] 2024-08-25T20:35:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:35:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 90259 2024-08-25T20:35:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:35:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 90261 2024-08-25T20:35:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:35:31.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 90263 2024-08-25T20:35:31.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:35:31.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 90265 2024-08-25T20:35:31.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=c61c5e0b398a48923f1ab3f321882580 2024-08-25T20:35:31.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-08-25T20:35:31.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ c61c5e0b398a48923f1ab3f321882580 != \c\6\1\c\5\e\0\b\3\9\8\a\4\8\9\2\3\f\1\a\b\3\f\3\2\1\8\8\2\5\8\0 ]] 2024-08-25T20:35:31.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:35:31.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 90267 2024-08-25T20:35:31.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:35:31.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 90269 2024-08-25T20:35:31.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:35:32.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-08-25T20:35:32.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:32.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:32.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:35:32.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:32.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:32.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:32.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:32.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:35:33.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:35:33.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e7e2ef0b-4966-416b-b3c4-b40984147b77 2024-08-25T20:35:33.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:35:33.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3095347.2,"bytes_per_snapshot":61906944.0,"last_snapshot_bytes":61906944,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"} 2024-08-25T20:35:33.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:35:33.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:35:28' 2024-08-25T20:35:33.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:35:33.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:35:33.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:35:33.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3095347.2,"bytes_per_snapshot":61906944.0,"last_snapshot_bytes":61906944,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"} 2024-08-25T20:35:33.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:35:33.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:33.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:35:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:35:33.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:35:33.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:35:33.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:35:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-08-25 20:35:28' 2024-08-25T20:35:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:35:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:35:28' 2024-08-25T20:35:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:33.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:33.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:35:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1e7e2ef0b-4966-416b-b3c4-b40984147b77up+replayingreplaying, {"bytes_per_second":3095347.2,"bytes_per_snapshot":61906944.0,"last_snapshot_bytes":61906944,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+replayingreplaying, {"bytes_per_second":3410124.8,"bytes_per_snapshot":68202496.0,"last_snapshot_bytes":68202496,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618078,"remote_snapshot_timestamp":1724618078,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+replayingreplaying, {"bytes_per_second":3130777.6,"bytes_per_snapshot":62615552.0,"last_snapshot_bytes":62615552,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+replayingreplaying, {"bytes_per_second":3401318.4,"bytes_per_snapshot":68026368.0,"last_snapshot_bytes":68026368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618076,"remote_snapshot_timestamp":1724618076,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+replayingreplaying, {"bytes_per_second":3381043.2,"bytes_per_snapshot":67620864.0,"last_snapshot_bytes":67620864,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+replayingreplaying, {"bytes_per_second":3408076.8,"bytes_per_snapshot":68161536.0,"last_snapshot_bytes":68161536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618071,"remote_snapshot_timestamp":1724618071,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+replayingreplaying, {"bytes_per_second":3334758.4,"bytes_per_snapshot":66695168.0,"last_snapshot_bytes":66695168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618075,"remote_snapshot_timestamp":1724618075,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+replayingreplaying, {"bytes_per_second":3237068.8,"bytes_per_snapshot":64741376.0,"last_snapshot_bytes":64741376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+replayingreplaying, {"bytes_per_second":3214950.4,"bytes_per_snapshot":64299008.0,"last_snapshot_bytes":64299008,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+replayingreplaying, {"bytes_per_second":3322880.0,"bytes_per_snapshot":66457600.0,"last_snapshot_bytes":66457600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28' 2024-08-25T20:35:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:35:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:35:33.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:33.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:35:33.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:35:33.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:33.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:35:33.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:33.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:33.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:33.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary1 2024-08-25T20:35:33.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:35:33.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:33.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:35:33.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-08-25T20:35:33.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-08-25T20:35:33.390 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:33.389+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4b95040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:35:33.399 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:33.389+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972c80 [3/e7e2ef0b-4966-416b-b3c4-b40984147b77] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:35:34.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:35:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-08-25T20:35:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:35:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:35:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary1 2024-08-25T20:35:34.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:34.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary1 2024-08-25T20:35:34.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:35:34.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:35:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-08-25T20:35:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:35:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:35:35.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:35:35.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a369576b-ec7f-4cd4-8942-e9c93da688f5 2024-08-25T20:35:35.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:35:35.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3130777.6,"bytes_per_snapshot":62615552.0,"last_snapshot_bytes":62615552,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"} 2024-08-25T20:35:35.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:35:35.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:35:28' 2024-08-25T20:35:35.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:35:35.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:35:35.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:35:35.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3130777.6,"bytes_per_snapshot":62615552.0,"last_snapshot_bytes":62615552,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"} 2024-08-25T20:35:35.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:35:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:35:35.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:35.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:35:35.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:35:35.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:35:35.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:35:35.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:35.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-08-25 20:35:28' 2024-08-25T20:35:35.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:35.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:35.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:35:35.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:35.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:35:28' 2024-08-25T20:35:35.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:35.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:35.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:35:35.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING941904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+replayingreplaying, {"bytes_per_second":3410124.8,"bytes_per_snapshot":68202496.0,"last_snapshot_bytes":68202496,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618078,"remote_snapshot_timestamp":1724618078,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary2a369576b-ec7f-4cd4-8942-e9c93da688f5up+replayingreplaying, {"bytes_per_second":3130777.6,"bytes_per_snapshot":62615552.0,"last_snapshot_bytes":62615552,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+replayingreplaying, {"bytes_per_second":3401318.4,"bytes_per_snapshot":68026368.0,"last_snapshot_bytes":68026368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618076,"remote_snapshot_timestamp":1724618076,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+replayingreplaying, {"bytes_per_second":3381043.2,"bytes_per_snapshot":67620864.0,"last_snapshot_bytes":67620864,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+replayingreplaying, {"bytes_per_second":3408076.8,"bytes_per_snapshot":68161536.0,"last_snapshot_bytes":68161536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618071,"remote_snapshot_timestamp":1724618071,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+replayingreplaying, {"bytes_per_second":3334758.4,"bytes_per_snapshot":66695168.0,"last_snapshot_bytes":66695168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618075,"remote_snapshot_timestamp":1724618075,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+replayingreplaying, {"bytes_per_second":3237068.8,"bytes_per_snapshot":64741376.0,"last_snapshot_bytes":64741376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+replayingreplaying, {"bytes_per_second":3214950.4,"bytes_per_snapshot":64299008.0,"last_snapshot_bytes":64299008,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+replayingreplaying, {"bytes_per_second":3322880.0,"bytes_per_snapshot":66457600.0,"last_snapshot_bytes":66457600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28' 2024-08-25T20:35:35.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:35:35.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:35:35.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:35.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:35:35.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:35:35.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:35.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:35:35.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:35.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary2 2024-08-25T20:35:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:35:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:35:35.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-08-25T20:35:35.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-08-25T20:35:36.137 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:36.136+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adf4c89c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:35:36.137 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:36.136+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972000 [3/a369576b-ec7f-4cd4-8942-e9c93da688f5] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:35:37.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:35:37.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-08-25T20:35:37.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:37.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:37.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:35:37.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:37.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:35:37.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary2 2024-08-25T20:35:37.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:37.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary2 2024-08-25T20:35:37.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:35:37.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:35:37.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-08-25T20:35:37.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:37.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:37.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:35:37.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:35:37.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-08-25T20:35:37.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:37.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:37.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:35:37.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:37.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:37.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:37.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:37.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:35:38.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-08-25T20:35:38.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:38.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:38.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:35:38.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:38.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:38.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:38.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:38.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:38.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:35:38.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:35:38.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9b637205-cc57-43e2-ab62-6a8b2b9c7b2d 2024-08-25T20:35:38.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:35:38.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3401318.4,"bytes_per_snapshot":68026368.0,"last_snapshot_bytes":68026368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618076,"remote_snapshot_timestamp":1724618076,"replay_state":"idle"} 2024-08-25T20:35:38.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:35:38.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:35:28' 2024-08-25T20:35:38.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:35:38.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:35:38.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:35:38.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3401318.4,"bytes_per_snapshot":68026368.0,"last_snapshot_bytes":68026368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618076,"remote_snapshot_timestamp":1724618076,"replay_state":"idle"} 2024-08-25T20:35:38.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:35:38.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:38.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:35:38.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:38.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:35:38.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:35:38.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:35:38.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:35:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:38.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-08-25 20:35:28' 2024-08-25T20:35:38.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:38.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:38.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:35:38.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:38.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:35:28' 2024-08-25T20:35:38.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:38.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:38.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:35:38.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING841904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+replayingreplaying, {"bytes_per_second":3410124.8,"bytes_per_snapshot":68202496.0,"last_snapshot_bytes":68202496,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618078,"remote_snapshot_timestamp":1724618078,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary39b637205-cc57-43e2-ab62-6a8b2b9c7b2dup+replayingreplaying, {"bytes_per_second":3401318.4,"bytes_per_snapshot":68026368.0,"last_snapshot_bytes":68026368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618076,"remote_snapshot_timestamp":1724618076,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+replayingreplaying, {"bytes_per_second":3381043.2,"bytes_per_snapshot":67620864.0,"last_snapshot_bytes":67620864,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+replayingreplaying, {"bytes_per_second":3408076.8,"bytes_per_snapshot":68161536.0,"last_snapshot_bytes":68161536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618071,"remote_snapshot_timestamp":1724618071,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+replayingreplaying, {"bytes_per_second":3334758.4,"bytes_per_snapshot":66695168.0,"last_snapshot_bytes":66695168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618075,"remote_snapshot_timestamp":1724618075,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+replayingreplaying, {"bytes_per_second":3237068.8,"bytes_per_snapshot":64741376.0,"last_snapshot_bytes":64741376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+replayingreplaying, {"bytes_per_second":3214950.4,"bytes_per_snapshot":64299008.0,"last_snapshot_bytes":64299008,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+replayingreplaying, {"bytes_per_second":3322880.0,"bytes_per_snapshot":66457600.0,"last_snapshot_bytes":66457600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28' 2024-08-25T20:35:38.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:35:38.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:35:38.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:38.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:35:38.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:35:38.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:38.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:35:38.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:38.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:38.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:38.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary3 2024-08-25T20:35:38.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:35:38.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:38.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:35:38.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-08-25T20:35:39.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-08-25T20:35:39.149 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:39.148+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4ed6d00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:35:39.149 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:39.148+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ae365fb80 [3/9b637205-cc57-43e2-ab62-6a8b2b9c7b2d] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:35:40.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:35:40.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-08-25T20:35:40.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:40.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:40.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:35:40.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:40.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:35:40.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary3 2024-08-25T20:35:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary3 2024-08-25T20:35:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:35:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:35:41.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-08-25T20:35:41.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:41.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:41.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:35:41.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:41.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:41.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:41.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:41.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:41.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:35:41.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:35:41.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b603d324-2f1a-4cd8-a553-2dc40e9b9994 2024-08-25T20:35:41.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:35:41.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3381043.2,"bytes_per_snapshot":67620864.0,"last_snapshot_bytes":67620864,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"} 2024-08-25T20:35:41.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:35:41.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:35:28' 2024-08-25T20:35:41.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:35:41.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:35:41.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:35:41.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3381043.2,"bytes_per_snapshot":67620864.0,"last_snapshot_bytes":67620864,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"} 2024-08-25T20:35:41.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:35:41.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:41.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:35:41.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:41.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:35:41.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:35:41.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:35:41.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:35:41.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:41.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-08-25 20:35:28' 2024-08-25T20:35:41.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:41.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:41.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:35:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:35:28' 2024-08-25T20:35:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:35:41.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING741904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+replayingreplaying, {"bytes_per_second":3410124.8,"bytes_per_snapshot":68202496.0,"last_snapshot_bytes":68202496,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618078,"remote_snapshot_timestamp":1724618078,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary4b603d324-2f1a-4cd8-a553-2dc40e9b9994up+replayingreplaying, {"bytes_per_second":3381043.2,"bytes_per_snapshot":67620864.0,"last_snapshot_bytes":67620864,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+replayingreplaying, {"bytes_per_second":3408076.8,"bytes_per_snapshot":68161536.0,"last_snapshot_bytes":68161536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618071,"remote_snapshot_timestamp":1724618071,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+replayingreplaying, {"bytes_per_second":3334758.4,"bytes_per_snapshot":66695168.0,"last_snapshot_bytes":66695168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618075,"remote_snapshot_timestamp":1724618075,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+replayingreplaying, {"bytes_per_second":3237068.8,"bytes_per_snapshot":64741376.0,"last_snapshot_bytes":64741376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+replayingreplaying, {"bytes_per_second":3214950.4,"bytes_per_snapshot":64299008.0,"last_snapshot_bytes":64299008,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+replayingreplaying, {"bytes_per_second":3322880.0,"bytes_per_snapshot":66457600.0,"last_snapshot_bytes":66457600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28' 2024-08-25T20:35:41.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:35:41.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:35:41.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:41.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:35:41.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:35:41.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:41.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:35:41.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:41.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:41.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:41.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary4 2024-08-25T20:35:41.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:35:41.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:41.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:35:41.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-08-25T20:35:41.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-08-25T20:35:42.056 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:42.056+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae33896c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:35:42.056 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:42.056+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973b80 [3/b603d324-2f1a-4cd8-a553-2dc40e9b9994] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:35:43.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:35:43.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-08-25T20:35:43.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:43.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:43.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:35:43.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:43.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:35:43.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary4 2024-08-25T20:35:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary4 2024-08-25T20:35:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:35:43.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:35:43.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-08-25T20:35:43.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:43.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:43.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:35:43.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:35:43.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-08-25T20:35:43.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:43.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:43.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:35:43.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:43.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:43.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:43.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:43.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:35:44.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-08-25T20:35:44.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:44.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:44.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:35:44.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:44.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:44.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:44.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:44.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:44.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:35:44.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:35:44.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 27def1ab-cb8e-459d-9676-6dc5bb3a2428 2024-08-25T20:35:44.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:35:44.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3408076.8,"bytes_per_snapshot":68161536.0,"last_snapshot_bytes":68161536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618071,"remote_snapshot_timestamp":1724618071,"replay_state":"idle"} 2024-08-25T20:35:44.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:35:44.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:35:28' 2024-08-25T20:35:44.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:35:44.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:35:44.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:35:44.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3408076.8,"bytes_per_snapshot":68161536.0,"last_snapshot_bytes":68161536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618071,"remote_snapshot_timestamp":1724618071,"replay_state":"idle"} 2024-08-25T20:35:44.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:35:44.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:44.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:35:44.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:44.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:35:44.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:35:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:35:44.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:35:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-08-25 20:35:28' 2024-08-25T20:35:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:35:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:35:28' 2024-08-25T20:35:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:44.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:35:44.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING641904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+replayingreplaying, {"bytes_per_second":3410124.8,"bytes_per_snapshot":68202496.0,"last_snapshot_bytes":68202496,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618078,"remote_snapshot_timestamp":1724618078,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary527def1ab-cb8e-459d-9676-6dc5bb3a2428up+replayingreplaying, {"bytes_per_second":3408076.8,"bytes_per_snapshot":68161536.0,"last_snapshot_bytes":68161536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618071,"remote_snapshot_timestamp":1724618071,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+replayingreplaying, {"bytes_per_second":3334758.4,"bytes_per_snapshot":66695168.0,"last_snapshot_bytes":66695168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618075,"remote_snapshot_timestamp":1724618075,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+replayingreplaying, {"bytes_per_second":3237068.8,"bytes_per_snapshot":64741376.0,"last_snapshot_bytes":64741376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+replayingreplaying, {"bytes_per_second":3214950.4,"bytes_per_snapshot":64299008.0,"last_snapshot_bytes":64299008,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+replayingreplaying, {"bytes_per_second":3322880.0,"bytes_per_snapshot":66457600.0,"last_snapshot_bytes":66457600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28' 2024-08-25T20:35:44.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:35:44.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:35:44.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:44.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:35:44.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:35:44.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:44.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:35:44.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:44.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:44.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:44.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary5 2024-08-25T20:35:44.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:35:44.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:44.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:35:44.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-08-25T20:35:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-08-25T20:35:45.192 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:45.192+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae36fe000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:35:45.192 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:45.192+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972780 [3/27def1ab-cb8e-459d-9676-6dc5bb3a2428] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:35:46.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:35:46.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-08-25T20:35:46.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:46.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:46.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:35:46.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:46.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:35:46.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary5 2024-08-25T20:35:46.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:46.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary5 2024-08-25T20:35:46.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:35:46.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:35:47.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-08-25T20:35:47.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:47.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:47.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:35:47.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:47.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:47.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:47.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:47.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:47.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:35:47.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:35:47.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 6c9ee9a6-a0b5-4565-947e-12f148eca87f 2024-08-25T20:35:47.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:35:47.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3334758.4,"bytes_per_snapshot":66695168.0,"last_snapshot_bytes":66695168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618075,"remote_snapshot_timestamp":1724618075,"replay_state":"idle"} 2024-08-25T20:35:47.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:35:47.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:35:28' 2024-08-25T20:35:47.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:35:47.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:35:47.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:35:47.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3334758.4,"bytes_per_snapshot":66695168.0,"last_snapshot_bytes":66695168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618075,"remote_snapshot_timestamp":1724618075,"replay_state":"idle"} 2024-08-25T20:35:47.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:35:47.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:47.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:35:47.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:47.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:35:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:35:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:35:47.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:35:47.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:47.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-08-25 20:35:28' 2024-08-25T20:35:47.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:47.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:47.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:35:47.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:47.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:35:28' 2024-08-25T20:35:47.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:47.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:47.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:35:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING541904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+replayingreplaying, {"bytes_per_second":3410124.8,"bytes_per_snapshot":68202496.0,"last_snapshot_bytes":68202496,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618078,"remote_snapshot_timestamp":1724618078,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary66c9ee9a6-a0b5-4565-947e-12f148eca87fup+replayingreplaying, {"bytes_per_second":3334758.4,"bytes_per_snapshot":66695168.0,"last_snapshot_bytes":66695168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618075,"remote_snapshot_timestamp":1724618075,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+replayingreplaying, {"bytes_per_second":3237068.8,"bytes_per_snapshot":64741376.0,"last_snapshot_bytes":64741376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+replayingreplaying, {"bytes_per_second":3214950.4,"bytes_per_snapshot":64299008.0,"last_snapshot_bytes":64299008,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+replayingreplaying, {"bytes_per_second":3322880.0,"bytes_per_snapshot":66457600.0,"last_snapshot_bytes":66457600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28' 2024-08-25T20:35:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:35:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:35:47.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:47.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:35:47.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:35:47.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:47.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:35:47.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:47.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:47.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:47.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary6 2024-08-25T20:35:47.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:35:47.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:47.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:35:47.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-08-25T20:35:47.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-08-25T20:35:47.979 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:47.978+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae36fe680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:35:47.979 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:47.978+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972f00 [3/6c9ee9a6-a0b5-4565-947e-12f148eca87f] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:35:49.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:35:49.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-08-25T20:35:49.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:49.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:49.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:35:49.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:49.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:35:49.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary6 2024-08-25T20:35:49.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:49.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary6 2024-08-25T20:35:49.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:35:49.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:35:49.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-08-25T20:35:49.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:35:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:35:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-08-25T20:35:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:35:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:49.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:35:50.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-08-25T20:35:50.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:50.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:50.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:35:50.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:50.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:50.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:50.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:50.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:50.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:35:50.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:35:50.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 31a35556-0c5d-41e5-9ae3-cd9b163ef396 2024-08-25T20:35:50.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:35:50.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3237068.8,"bytes_per_snapshot":64741376.0,"last_snapshot_bytes":64741376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"} 2024-08-25T20:35:50.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:35:50.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:35:28' 2024-08-25T20:35:50.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:35:50.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:35:50.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:35:50.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3237068.8,"bytes_per_snapshot":64741376.0,"last_snapshot_bytes":64741376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"} 2024-08-25T20:35:50.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:35:50.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:50.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:35:50.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:50.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:35:50.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:35:50.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:35:50.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:35:50.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:50.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-08-25 20:35:28' 2024-08-25T20:35:50.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:50.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:50.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:35:50.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:50.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:35:28' 2024-08-25T20:35:50.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:50.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:50.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:35:50.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING441904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+replayingreplaying, {"bytes_per_second":3410124.8,"bytes_per_snapshot":68202496.0,"last_snapshot_bytes":68202496,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618078,"remote_snapshot_timestamp":1724618078,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary731a35556-0c5d-41e5-9ae3-cd9b163ef396up+replayingreplaying, {"bytes_per_second":3237068.8,"bytes_per_snapshot":64741376.0,"last_snapshot_bytes":64741376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618074,"remote_snapshot_timestamp":1724618074,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+replayingreplaying, {"bytes_per_second":3214950.4,"bytes_per_snapshot":64299008.0,"last_snapshot_bytes":64299008,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+replayingreplaying, {"bytes_per_second":3322880.0,"bytes_per_snapshot":66457600.0,"last_snapshot_bytes":66457600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28' 2024-08-25T20:35:50.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:35:50.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:35:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:50.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:35:50.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:35:50.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:50.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:35:50.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:50.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:50.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:50.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary7 2024-08-25T20:35:50.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:35:50.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:50.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:35:50.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-08-25T20:35:50.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-08-25T20:35:50.979 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:50.978+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3662ea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:35:50.979 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:50.978+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973180 [3/31a35556-0c5d-41e5-9ae3-cd9b163ef396] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:35:52.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:35:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-08-25T20:35:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:35:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:35:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary7 2024-08-25T20:35:52.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:52.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary7 2024-08-25T20:35:52.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:35:52.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:35:52.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-08-25T20:35:52.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:52.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:52.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:35:52.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:35:52.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-08-25T20:35:52.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:52.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:52.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:35:52.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:52.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:52.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:52.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:52.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:35:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-08-25T20:35:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:35:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:53.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:53.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:35:53.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:35:53.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e5f22a6f-8b4b-42b8-b3e5-b053140e92e2 2024-08-25T20:35:53.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:35:53.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3214950.4,"bytes_per_snapshot":64299008.0,"last_snapshot_bytes":64299008,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"} 2024-08-25T20:35:53.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:35:53.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:35:28' 2024-08-25T20:35:53.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:35:53.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:35:53.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:35:53.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3214950.4,"bytes_per_snapshot":64299008.0,"last_snapshot_bytes":64299008,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"} 2024-08-25T20:35:53.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:35:53.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:53.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:35:53.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:53.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:35:53.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:35:53.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:35:53.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:35:53.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:53.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-08-25 20:35:28' 2024-08-25T20:35:53.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:53.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:53.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:35:53.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:53.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:35:28' 2024-08-25T20:35:53.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:53.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:53.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:35:53.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING341904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+replayingreplaying, {"bytes_per_second":3410124.8,"bytes_per_snapshot":68202496.0,"last_snapshot_bytes":68202496,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618078,"remote_snapshot_timestamp":1724618078,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary8e5f22a6f-8b4b-42b8-b3e5-b053140e92e2up+replayingreplaying, {"bytes_per_second":3214950.4,"bytes_per_snapshot":64299008.0,"last_snapshot_bytes":64299008,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+replayingreplaying, {"bytes_per_second":3322880.0,"bytes_per_snapshot":66457600.0,"last_snapshot_bytes":66457600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28' 2024-08-25T20:35:53.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:35:53.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:35:53.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:53.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:35:53.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:35:53.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:53.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:35:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary8 2024-08-25T20:35:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:35:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:35:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-08-25T20:35:53.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-08-25T20:35:53.988 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:53.988+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4e6c4e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:35:53.989 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:53.988+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972280 [3/e5f22a6f-8b4b-42b8-b3e5-b053140e92e2] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:35:55.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:35:55.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-08-25T20:35:55.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:55.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:55.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:35:55.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:55.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:35:55.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary8 2024-08-25T20:35:55.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:55.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary8 2024-08-25T20:35:55.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:35:55.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:35:55.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-08-25T20:35:55.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:55.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:55.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:35:55.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:35:55.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-08-25T20:35:55.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:55.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:55.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:35:55.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:55.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:55.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:55.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:55.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:35:56.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-08-25T20:35:56.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:56.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:56.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:35:56.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:56.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:56.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:56.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:56.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:56.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:35:56.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:35:56.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b109da61-be86-4ef4-a8a7-95d5ee0e539e 2024-08-25T20:35:56.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:35:56.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3322880.0,"bytes_per_snapshot":66457600.0,"last_snapshot_bytes":66457600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"} 2024-08-25T20:35:56.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:35:56.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:35:28' 2024-08-25T20:35:56.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:35:56.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:35:56.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:35:56.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3322880.0,"bytes_per_snapshot":66457600.0,"last_snapshot_bytes":66457600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"} 2024-08-25T20:35:56.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:35:56.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:56.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:35:56.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:56.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:35:56.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:35:56.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:35:56.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:35:56.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:56.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-08-25 20:35:28' 2024-08-25T20:35:56.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:56.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:56.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:35:56.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:56.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:35:28' 2024-08-25T20:35:56.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:56.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:56.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:35:56.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING241904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+replayingreplaying, {"bytes_per_second":3410124.8,"bytes_per_snapshot":68202496.0,"last_snapshot_bytes":68202496,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618078,"remote_snapshot_timestamp":1724618078,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28image-primary9b109da61-be86-4ef4-a8a7-95d5ee0e539eup+replayingreplaying, {"bytes_per_second":3322880.0,"bytes_per_snapshot":66457600.0,"last_snapshot_bytes":66457600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618077,"remote_snapshot_timestamp":1724618077,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:28' 2024-08-25T20:35:56.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:35:56.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:35:56.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:28' 2024-08-25T20:35:56.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:35:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:35:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:56.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:35:56.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:56.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary9 2024-08-25T20:35:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:35:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:35:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-08-25T20:35:56.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-08-25T20:35:56.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:56.982+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3503d40 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:35:56.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:56.982+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973680 [3/b109da61-be86-4ef4-a8a7-95d5ee0e539e] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:35:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:35:58.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-08-25T20:35:58.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:58.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:58.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:35:58.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:58.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:35:58.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary9 2024-08-25T20:35:58.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:58.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary9 2024-08-25T20:35:58.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:35:58.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:35:58.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-08-25T20:35:58.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:58.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:58.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:35:58.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:35:58.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-08-25T20:35:58.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:58.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:58.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:35:58.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:58.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:58.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:58.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:35:58.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:35:59.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-08-25T20:35:59.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:59.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:59.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:35:59.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:59.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:35:59.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:35:59.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:59.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:59.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:35:59.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:35:59.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 371a44ff-21a7-4f10-bb6e-04ba829deb66 2024-08-25T20:35:59.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:35:59.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":68202496.0,"last_snapshot_bytes":68202496,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618078,"remote_snapshot_timestamp":1724618078,"replay_state":"idle"} 2024-08-25T20:35:59.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:35:59.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:35:58' 2024-08-25T20:35:59.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:35:59.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:35:59.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:35:59.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":68202496.0,"last_snapshot_bytes":68202496,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618078,"remote_snapshot_timestamp":1724618078,"replay_state":"idle"} 2024-08-25T20:35:59.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:35:59.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:59.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:35:59.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:59.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:35:59.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:35:59.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:35:59.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:35:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:58' 2024-08-25T20:35:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-08-25 20:35:58' 2024-08-25T20:35:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:35:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:35:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:35:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:35:58' 2024-08-25T20:35:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:35:59.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:35:59.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:35:59.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING141904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10371a44ff-21a7-4f10-bb6e-04ba829deb66up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":68202496.0,"last_snapshot_bytes":68202496,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618078,"remote_snapshot_timestamp":1724618078,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:35:58' 2024-08-25T20:35:59.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:35:59.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:35:59.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:35:58' 2024-08-25T20:35:59.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:35:59.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:35:59.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:35:59.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:35:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:35:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:35:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary10 2024-08-25T20:35:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:35:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:35:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:35:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-08-25T20:35:59.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-08-25T20:35:59.987 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:59.985+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae36fe820 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:35:59.987 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:35:59.985+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973900 [3/371a44ff-21a7-4f10-bb6e-04ba829deb66] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:36:01.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:36:01.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-08-25T20:36:01.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:01.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:01.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:01.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:36:01.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:36:01.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary10 2024-08-25T20:36:01.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:36:01.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary10 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..10} 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS=() 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS=() 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS=() 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS=() 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RET=0 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary1 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:36:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:36:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:36:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-08-25T20:36:01.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:36:01.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:01.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:36:01.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:36:01.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-08-25T20:36:01.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:01.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:01.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:01.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:01.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-08-25T20:36:02.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:36:02.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-08-25T20:36:02.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary1': 2024-08-25T20:36:02.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:36:02.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:36:02.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:36:02.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2ec3ce0f9229 2024-08-25T20:36:02.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2ec3ce0f9229 2024-08-25T20:36:02.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:36:02.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:36:02.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:36:02.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:36:02.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:36:01 2024 2024-08-25T20:36:02.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:36:01 2024 2024-08-25T20:36:02.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:36:01 2024 2024-08-25T20:36:02.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:36:02.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:36:02.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 8d09bd3d-e631-4259-85a7-7df19d5d05a5 2024-08-25T20:36:02.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:36:02.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:36:02.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:36:02.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-08-25T20:36:02.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd0 2024-08-25T20:36:02.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:36:02.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-08-25T20:36:02.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:36:02.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:36:02.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:36:02.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 93e3b85e-5bf2-4a82-8151-902fa0f4965f 2024-08-25T20:36:02.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:36:02.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:36:02.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:02.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:36:02.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:36:02.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:36:02.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:36:02.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:02.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary1 2024-08-25T20:36:02.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-08-25T20:36:02.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:36:02.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary1 2024-08-25T20:36:02.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:36:02.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-08-25T20:36:02.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:02.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:02.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:02.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:02.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:02.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:02.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:36:02.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:02.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:02.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:02.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:36:02.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:36:02.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary2 2024-08-25T20:36:02.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-08-25T20:36:02.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary1 2024-08-25T20:36:02.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:02.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:02.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:02.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:02.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:02.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:02.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:02.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:36:02.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:02.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-08-25T20:36:02.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:02.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary1 2024-08-25T20:36:02.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:02.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:02.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:02.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:02.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-08-25T20:36:02.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:02.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:36:02.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:36:02.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:36:02.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:02.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-08-25T20:36:03.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:36:03.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:03.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:36:03.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:36:03.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-08-25T20:36:03.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:03.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:03.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:03.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:03.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-08-25T20:36:03.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:36:03.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-08-25T20:36:03.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:36:03.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1984 2024-08-25T20:36:03.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:03.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-08-25T20:36:03.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary2': 2024-08-25T20:36:03.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:36:03.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:36:03.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:36:03.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2ed31c69595e 2024-08-25T20:36:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2ed31c69595e 2024-08-25T20:36:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:36:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:36:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:36:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:36:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:36:03 2024 2024-08-25T20:36:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:36:03 2024 2024-08-25T20:36:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:36:03 2024 2024-08-25T20:36:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:36:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:36:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: ae736451-33b4-4280-b078-499058ffab98 2024-08-25T20:36:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:36:03.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:36:03.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:36:03.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-08-25T20:36:03.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd1 2024-08-25T20:36:03.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:36:03.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-08-25T20:36:03.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:36:03.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:36:03.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:36:03.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 34085088-4f85-4093-8d46-f91ca57acaa0 2024-08-25T20:36:03.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:36:03.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:36:03.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:03.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:36:03.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:36:03.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:36:03.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:36:03.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:03.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary2 2024-08-25T20:36:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-08-25T20:36:03.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:36:03.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary2 2024-08-25T20:36:03.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:36:03.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-08-25T20:36:03.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:03.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:03.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:03.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:03.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:03.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:03.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:03.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:03.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:03.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:36:03.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:36:03.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:36:03.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary3 2024-08-25T20:36:03.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-08-25T20:36:03.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:03.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:03.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:36:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-08-25T20:36:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary2 2024-08-25T20:36:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:36:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:36:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:36:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary2 2024-08-25T20:36:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-08-25T20:36:03.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-08-25T20:36:03.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:36:03.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:03.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:36:03.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:36:03.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-08-25T20:36:03.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:03.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:03.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:03.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:03.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-08-25T20:36:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:36:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-08-25T20:36:04.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:36:04.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1986 2024-08-25T20:36:04.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-08-25T20:36:04.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:04.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary3': 2024-08-25T20:36:04.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:36:04.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:36:04.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:36:04.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2ee2afc44fa5 2024-08-25T20:36:04.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2ee2afc44fa5 2024-08-25T20:36:04.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:36:04.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:36:04.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:36:04.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:36:04.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:36:03 2024 2024-08-25T20:36:04.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:36:03 2024 2024-08-25T20:36:04.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:36:03 2024 2024-08-25T20:36:04.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:36:04.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:36:04.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 629954bc-57dc-455c-84fe-8764b2a92c2e 2024-08-25T20:36:04.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:36:04.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:36:04.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:36:04.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-08-25T20:36:04.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd2 2024-08-25T20:36:04.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:36:04.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-08-25T20:36:04.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:36:04.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:36:04.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:36:04.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: ae436a6f-36fe-4236-93ea-74c9235b42c1 2024-08-25T20:36:04.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:36:04.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:36:04.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:04.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:36:04.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:36:04.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:36:04.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:36:04.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:04.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary3 2024-08-25T20:36:04.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-08-25T20:36:04.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:36:04.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary3 2024-08-25T20:36:04.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:36:04.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-08-25T20:36:04.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:04.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:36:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:04.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:36:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:36:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:36:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:36:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary4 2024-08-25T20:36:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-08-25T20:36:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary3 2024-08-25T20:36:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:04.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:04.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:04.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:36:04.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:04.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-08-25T20:36:04.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary3 2024-08-25T20:36:04.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:04.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:04.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:04.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:04.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:04.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:04.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-08-25T20:36:04.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:36:04.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:36:04.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:36:04.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:04.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-08-25T20:36:05.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:36:05.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:05.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:36:05.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:36:05.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-08-25T20:36:05.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:05.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:05.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:05.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:05.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-08-25T20:36:05.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:36:05.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-08-25T20:36:05.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:36:05.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1988 2024-08-25T20:36:05.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-08-25T20:36:05.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:05.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary4': 2024-08-25T20:36:05.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:36:05.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:36:05.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:36:05.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2ef1f4595518 2024-08-25T20:36:05.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2ef1f4595518 2024-08-25T20:36:05.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:36:05.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:36:05.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:36:05.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:36:05.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:36:04 2024 2024-08-25T20:36:05.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:36:04 2024 2024-08-25T20:36:05.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:36:04 2024 2024-08-25T20:36:05.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:36:05.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:36:05.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: d8ea85d6-0a14-489d-88c9-253afe4f3a5c 2024-08-25T20:36:05.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:36:05.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:36:05.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:36:05.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-08-25T20:36:05.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd3 2024-08-25T20:36:05.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:36:05.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-08-25T20:36:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:36:05.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:36:05.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:36:05.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: c229200f-2c0b-490d-ba2a-5498cb3e6028 2024-08-25T20:36:05.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:36:05.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:36:05.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:05.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:36:05.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:36:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:36:05.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:05.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary4 2024-08-25T20:36:05.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-08-25T20:36:05.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:36:05.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary4 2024-08-25T20:36:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:36:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-08-25T20:36:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:36:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:36:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:36:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:36:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:36:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary4 2024-08-25T20:36:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary5 2024-08-25T20:36:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-08-25T20:36:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:36:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary4 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:36:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:05.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-08-25T20:36:06.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:36:06.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:06.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:36:06.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:36:06.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-08-25T20:36:06.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:06.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:06.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:06.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:06.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-08-25T20:36:06.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:36:06.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-08-25T20:36:06.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:06.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:06.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:06.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:06.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:06.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:06.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:36:06.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1990 2024-08-25T20:36:06.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-08-25T20:36:06.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary5': 2024-08-25T20:36:06.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:36:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:36:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:36:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2f003f2649cf 2024-08-25T20:36:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2f003f2649cf 2024-08-25T20:36:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:36:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:36:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:36:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:36:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:36:06 2024 2024-08-25T20:36:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:36:06 2024 2024-08-25T20:36:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:36:06 2024 2024-08-25T20:36:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:36:06.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:36:06.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 8ea7a427-cfbf-4f64-b72c-363c5306e09f 2024-08-25T20:36:06.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:36:06.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:36:06.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:36:06.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-08-25T20:36:06.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd4 2024-08-25T20:36:06.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:36:06.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-08-25T20:36:06.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:36:06.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:36:06.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:36:06.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: aa476310-2a08-4e32-b95d-3cdc146eef94 2024-08-25T20:36:06.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:36:06.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:36:06.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:06.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:36:06.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:36:06.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:36:06.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:36:06.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:06.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary5 2024-08-25T20:36:06.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-08-25T20:36:07.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:36:07.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary5 2024-08-25T20:36:07.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:36:07.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-08-25T20:36:07.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:07.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:07.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:07.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:07.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary6 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary5 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary5 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:07.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:07.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:36:07.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:36:07.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:36:07.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:07.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-08-25T20:36:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:36:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:07.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:36:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:36:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-08-25T20:36:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:07.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-08-25T20:36:07.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:36:07.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-08-25T20:36:07.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:07.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:07.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1992 2024-08-25T20:36:07.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:36:07.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1993 2024-08-25T20:36:07.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:07.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:07.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-08-25T20:36:07.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary6': 2024-08-25T20:36:07.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2f12dd546e1e 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2f12dd546e1e 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:36:07 2024 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:36:07 2024 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:36:07 2024 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:36:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 7a92202a-68ae-4d01-928a-945468468e5c 2024-08-25T20:36:07.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:36:07.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:36:07.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:36:07.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-08-25T20:36:07.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd5 2024-08-25T20:36:07.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:36:07.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-08-25T20:36:07.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:36:07.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:36:07.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:36:07.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: d1ea117d-14b4-4e5f-a615-300328224bec 2024-08-25T20:36:07.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:36:07.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:36:07.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:07.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:36:07.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:36:07.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:36:08.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:36:08.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary6 2024-08-25T20:36:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-08-25T20:36:08.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:36:08.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary6 2024-08-25T20:36:08.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:36:08.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-08-25T20:36:08.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary7 2024-08-25T20:36:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary6 2024-08-25T20:36:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-08-25T20:36:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:36:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-08-25T20:36:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary6 2024-08-25T20:36:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-08-25T20:36:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:36:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:36:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:36:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-08-25T20:36:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:36:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:36:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:36:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-08-25T20:36:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-08-25T20:36:08.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:36:08.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-08-25T20:36:08.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:08.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:36:08.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:08.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:08.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:08.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:36:08.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1996 2024-08-25T20:36:08.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:08.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1995 2024-08-25T20:36:08.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:09.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:36:09.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1998 2024-08-25T20:36:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:36:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:36:09.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:09.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-08-25T20:36:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary7': 2024-08-25T20:36:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:36:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:36:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2f2388a53b9b 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2f2388a53b9b 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:36:08 2024 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:36:08 2024 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:36:08 2024 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 7da99107-e1ee-4201-a999-13a8eec52934 2024-08-25T20:36:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:36:09.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:36:09.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:36:09.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-08-25T20:36:09.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd6 2024-08-25T20:36:09.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:36:09.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-08-25T20:36:09.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:36:09.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:36:09.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:36:09.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: cc47fe2a-a129-4198-80a7-9a3f9806f09a 2024-08-25T20:36:09.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:36:09.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:36:09.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:36:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:36:09.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:36:09.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:36:09.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:09.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary7 2024-08-25T20:36:09.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-08-25T20:36:10.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:36:10.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary7 2024-08-25T20:36:10.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:36:10.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-08-25T20:36:10.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:36:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:36:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary8 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary7 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:10.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary7 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:36:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-08-25T20:36:10.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:36:10.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:10.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:36:10.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:36:10.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-08-25T20:36:10.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:10.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:10.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:10.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:10.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-08-25T20:36:10.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:36:10.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-08-25T20:36:10.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:36:10.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2000 2024-08-25T20:36:10.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-08-25T20:36:10.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 1999 2024-08-25T20:36:10.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:10.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:10.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:10.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:10.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:10.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:10.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:10.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:10.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:10.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:36:10.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:10.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:10.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:10.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:36:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary8': 2024-08-25T20:36:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:36:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:36:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:36:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2f38fb7fc69d 2024-08-25T20:36:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2f38fb7fc69d 2024-08-25T20:36:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:36:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:36:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:36:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:36:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:36:10 2024 2024-08-25T20:36:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:36:10 2024 2024-08-25T20:36:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:36:10 2024 2024-08-25T20:36:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:36:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:36:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 72b47598-0266-4d1c-81e0-f751f633b29c 2024-08-25T20:36:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:36:10.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:36:10.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:36:10.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-08-25T20:36:10.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd7 2024-08-25T20:36:10.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:36:10.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-08-25T20:36:10.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:36:10.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:36:10.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:36:10.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 35200957-b5c0-4105-920c-9fb37a1ae51b 2024-08-25T20:36:10.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:36:10.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:36:10.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:10.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:36:10.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:36:10.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:36:10.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:36:10.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:10.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary8 2024-08-25T20:36:11.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-08-25T20:36:11.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:36:11.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary8 2024-08-25T20:36:11.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:36:11.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-08-25T20:36:11.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:11.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:36:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:36:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:36:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:36:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary9 2024-08-25T20:36:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary8 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary8 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-08-25T20:36:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:36:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:36:11.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:36:11.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:11.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-08-25T20:36:11.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:36:11.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:11.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:36:11.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:36:11.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-08-25T20:36:11.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:11.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:11.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:11.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:11.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-08-25T20:36:11.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:36:11.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-08-25T20:36:11.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:36:11.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2003 2024-08-25T20:36:11.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2004 2024-08-25T20:36:11.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2002 2024-08-25T20:36:11.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:11.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:11.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:11.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-08-25T20:36:11.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:11.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:36:11.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:11.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:11.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:11.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:36:11.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:11.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:11.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:11.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:11.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:11.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:11.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary9': 2024-08-25T20:36:11.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:36:11.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2f4daad95bed 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2f4daad95bed 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:36:11 2024 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:36:11 2024 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:36:11 2024 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:36:11.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: c3d225f7-8cf1-4273-85ce-9973dfd95e9a 2024-08-25T20:36:11.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:36:11.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:36:11.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:36:11.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-08-25T20:36:11.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd8 2024-08-25T20:36:11.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:36:11.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-08-25T20:36:11.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:36:11.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:36:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:36:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 55bf3082-8dbf-412c-9c2f-cd37a4b21d52 2024-08-25T20:36:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:36:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:36:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:36:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:36:11.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:36:11.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:36:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:11.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary9 2024-08-25T20:36:11.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-08-25T20:36:11.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:36:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary9 2024-08-25T20:36:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:36:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-08-25T20:36:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary10 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-08-25T20:36:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary9 2024-08-25T20:36:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:36:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary9 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:36:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-08-25T20:36:12.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:36:12.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:12.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:36:12.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:36:12.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-08-25T20:36:12.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:12.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:12.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:12.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:36:12.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-08-25T20:36:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:36:12.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-08-25T20:36:12.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2007 2024-08-25T20:36:12.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:12.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:36:12.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:12.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:12.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:12.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:36:12.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:36:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2008 2024-08-25T20:36:12.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2006 2024-08-25T20:36:12.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:12.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:12.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:12.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-08-25T20:36:12.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary10': 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 2f623aec744e 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.2f623aec744e 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:36:12 2024 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:36:12 2024 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:36:12 2024 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:36:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: c1d9ccc9-fc58-49ab-9a15-08aef159ef5f 2024-08-25T20:36:12.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:36:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:36:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:36:12.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-08-25T20:36:12.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd9 2024-08-25T20:36:12.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:36:12.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-08-25T20:36:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:36:12.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:36:12.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:36:12.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 1d1f24df-7de0-45d9-8c0b-b84fcd413c15 2024-08-25T20:36:12.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:36:12.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:36:12.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:12.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:36:12.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:36:12.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:36:12.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:36:12.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:36:12.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary10 2024-08-25T20:36:12.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-08-25T20:36:13.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:36:13.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary10 2024-08-25T20:36:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:36:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-08-25T20:36:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:36:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:36:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:36:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:36:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 94835 2024-08-25T20:36:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary10 2024-08-25T20:36:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary10 2024-08-25T20:36:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-08-25T20:36:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:36:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-08-25T20:36:13.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:13.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:36:13.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:13.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:13.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:13.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:36:13.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:13.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:36:13.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:13.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:13.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:13.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:36:13.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2010 2024-08-25T20:36:13.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2011 2024-08-25T20:36:13.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:13.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:14.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:14.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:14.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:14.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:14.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:14.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:14.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:36:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:36:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:36:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:36:14.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2012 2024-08-25T20:36:14.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2013 2024-08-25T20:36:14.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:14.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:15.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:15.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:15.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:15.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:15.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:15.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:15.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:15.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:36:15.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:15.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:15.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:15.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:36:15.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:15.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:36:15.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:15.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:15.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:15.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:36:15.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2016 2024-08-25T20:36:15.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2014 2024-08-25T20:36:15.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:15.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2015 2024-08-25T20:36:15.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:15.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:16.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2019 2024-08-25T20:36:16.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2017 2024-08-25T20:36:16.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2018 2024-08-25T20:36:16.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:16.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:16.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:16.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:16.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:36:16.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:16.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:16.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:16.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:36:16.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:16.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:36:16.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:16.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:16.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:16.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:36:17.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:17.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:36:17.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:17.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:17.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:17.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:36:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:36:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:36:17.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2021 2024-08-25T20:36:17.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2020 2024-08-25T20:36:17.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:17.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:36:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:36:18.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2023 2024-08-25T20:36:18.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2022 2024-08-25T20:36:18.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:18.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:36:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:36:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:18.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:19.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2025 2024-08-25T20:36:19.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:19.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:19.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2024 2024-08-25T20:36:19.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:19.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:19.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:19.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:19.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2026 2024-08-25T20:36:19.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:19.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:36:19.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:19.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:19.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:19.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:36:19.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:19.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:19.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:36:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:36:20.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2028 2024-08-25T20:36:20.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2029 2024-08-25T20:36:20.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:20.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:20.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2027 2024-08-25T20:36:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:20.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:20.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:36:20.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:20.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:20.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:20.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:36:20.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:20.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:36:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:36:21.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2030 2024-08-25T20:36:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:36:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:21.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:36:21.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:21.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:36:21.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:21.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:21.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:21.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:36:21.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2031 2024-08-25T20:36:21.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:21.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2032 2024-08-25T20:36:22.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2033 2024-08-25T20:36:22.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:22.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:22.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:22.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:36:22.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:22.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:22.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:22.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:36:22.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:22.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:36:22.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:22.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:22.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:22.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:36:22.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:22.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:22.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:22.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:22.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:22.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:23.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2034 2024-08-25T20:36:23.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:23.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2035 2024-08-25T20:36:23.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:23.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:23.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:36:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:36:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:36:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:36:23.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:23.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:23.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:23.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:23.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:23.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:24.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2037 2024-08-25T20:36:24.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2036 2024-08-25T20:36:24.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:24.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2038 2024-08-25T20:36:24.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:24.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:24.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:24.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:36:24.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:24.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:24.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:24.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:36:24.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:24.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:36:24.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:24.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:24.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:24.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:36:25.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:25.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:36:25.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:25.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:25.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:25.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:36:25.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:25.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:36:25.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:25.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:25.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:25.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:36:25.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2040 2024-08-25T20:36:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:26.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2039 2024-08-25T20:36:26.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:26.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2041 2024-08-25T20:36:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:36:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:26.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:26.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:36:26.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:26.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:26.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:36:26.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:26.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:26.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:26.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:36:26.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2042 2024-08-25T20:36:26.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:27.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2043 2024-08-25T20:36:27.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:27.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2044 2024-08-25T20:36:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:36:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:36:27.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:27.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:36:27.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:27.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:27.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:27.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:36:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2045 2024-08-25T20:36:28.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2047 2024-08-25T20:36:28.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:28.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:28.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:28.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:36:28.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:28.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:28.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:28.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:36:29.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:29.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:29.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:29.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:29.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:29.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2046 2024-08-25T20:36:29.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:29.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2048 2024-08-25T20:36:29.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2049 2024-08-25T20:36:29.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:36:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:36:29.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:29.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:29.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:29.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:36:29.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:29.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:29.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:29.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:36:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2050 2024-08-25T20:36:30.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:30.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:36:30.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:30.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:30.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:30.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:36:30.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:36:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:36:31.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:31.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:36:31.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:31.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:31.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:31.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:36:31.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:31.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:36:31.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:31.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:31.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:31.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:36:32.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2051 2024-08-25T20:36:32.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2052 2024-08-25T20:36:32.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:32.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:32.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:32.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:32.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:32.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:32.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:32.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:36:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:36:32.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:32.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:36:32.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:32.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:32.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:32.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:36:33.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:33.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:33.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:33.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:33.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:33.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:33.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2054 2024-08-25T20:36:33.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2053 2024-08-25T20:36:33.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:34.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2056 2024-08-25T20:36:34.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2055 2024-08-25T20:36:34.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:35.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:35.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:36:35.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:35.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:35.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:35.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:36:35.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:35.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:36:35.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:35.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:35.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:35.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:36:35.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2058 2024-08-25T20:36:35.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2059 2024-08-25T20:36:35.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2057 2024-08-25T20:36:35.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:35.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:35.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:36.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2060 2024-08-25T20:36:36.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:36.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2062 2024-08-25T20:36:36.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2061 2024-08-25T20:36:36.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:36.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:36.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:36:36.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:36.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:36.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:36.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:36:36.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:36.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:36:36.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:36.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:36.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:36.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:36:37.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2064 2024-08-25T20:36:37.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:37.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:36:37.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:37.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:37.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:36:37.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2063 2024-08-25T20:36:37.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:37.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:37.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:37.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:36:37.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:37.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:37.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:37.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:36:38.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2065 2024-08-25T20:36:38.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2066 2024-08-25T20:36:38.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:38.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:38.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:38.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:38.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:38.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:38.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:38.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:38.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:38.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:36:38.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:38.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:38.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:38.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:36:38.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:38.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:36:38.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:38.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:38.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:38.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:36:39.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:39.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:39.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:39.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:39.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:39.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:39.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:39.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:36:39.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:39.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:39.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:39.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:36:39.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:36:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:36:39.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2070 2024-08-25T20:36:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2069 2024-08-25T20:36:39.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2067 2024-08-25T20:36:39.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:39.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:39.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2068 2024-08-25T20:36:39.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:39.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:40.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:40.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:36:40.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:40.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:40.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:40.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:36:40.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:40.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:36:40.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:40.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:40.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:40.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:36:41.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2072 2024-08-25T20:36:41.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2071 2024-08-25T20:36:41.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:41.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:41.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:41.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:36:41.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:41.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:41.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:41.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:36:41.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:41.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:36:41.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:41.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:41.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:41.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:36:42.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2074 2024-08-25T20:36:42.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2073 2024-08-25T20:36:42.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:42.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:36:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:42.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:42.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:36:42.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:42.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:42.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:42.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:42.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:42.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:42.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:42.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:42.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:36:42.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:42.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:42.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:42.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:36:43.629 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2076 2024-08-25T20:36:43.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:44.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2075 2024-08-25T20:36:44.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:44.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:36:44.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:44.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:44.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:44.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:36:44.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:44.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:36:44.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:44.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:44.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:44.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:36:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:44.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2080 2024-08-25T20:36:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2079 2024-08-25T20:36:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:44.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:45.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2077 2024-08-25T20:36:45.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2078 2024-08-25T20:36:45.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:45.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:45.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:36:45.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:45.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:45.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:45.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:36:45.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:45.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:36:45.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:45.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:45.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:45.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:36:45.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:46.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:46.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:36:46.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:46.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:46.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:46.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:36:47.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2082 2024-08-25T20:36:47.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2081 2024-08-25T20:36:47.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:47.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:47.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:47.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:36:47.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:47.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:47.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:47.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:36:47.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:47.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:36:47.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:47.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:47.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:47.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:36:47.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:47.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:36:47.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:47.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:47.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:47.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:36:48.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2083 2024-08-25T20:36:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2084 2024-08-25T20:36:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:48.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:48.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:48.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:48.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:48.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:48.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:48.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:48.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2086 2024-08-25T20:36:48.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:49.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2085 2024-08-25T20:36:49.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2090 2024-08-25T20:36:49.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2089 2024-08-25T20:36:49.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:49.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:50.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2087 2024-08-25T20:36:50.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2088 2024-08-25T20:36:50.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:50.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:36:50.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:50.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:50.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:50.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:36:50.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:50.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:50.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:50.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:36:50.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:50.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:50.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:50.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:36:51.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2092 2024-08-25T20:36:51.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2091 2024-08-25T20:36:51.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:51.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:36:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:36:51.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:51.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:36:51.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:51.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:51.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:51.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:36:51.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:51.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:36:51.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:51.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:51.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:51.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:36:52.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:52.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:36:52.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:52.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:52.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:52.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:36:52.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2094 2024-08-25T20:36:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:52.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:52.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:52.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:52.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:52.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:52.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:52.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:52.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:52.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:52.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:52.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:52.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:52.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2093 2024-08-25T20:36:52.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:53.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:36:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:36:53.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:53.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:36:53.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:53.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:53.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:53.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:36:53.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2096 2024-08-25T20:36:53.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:54.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2095 2024-08-25T20:36:54.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:36:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:36:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:36:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:36:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:36:54.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:36:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:54.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2099 2024-08-25T20:36:54.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:54.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2100 2024-08-25T20:36:54.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:55.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2097 2024-08-25T20:36:55.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2098 2024-08-25T20:36:55.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:55.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:55.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:55.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:36:55.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:55.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:55.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:36:55.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:36:55.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:55.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:36:55.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:55.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:55.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:36:55.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:36:56.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:56.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:36:56.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:56.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:56.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:36:56.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:36:57.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2102 2024-08-25T20:36:57.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2101 2024-08-25T20:36:57.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:57.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:57.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:36:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:36:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:36:57.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:57.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:36:57.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:57.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:57.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:36:57.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:36:57.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:57.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:36:57.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:57.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:57.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:36:57.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:36:58.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2104 2024-08-25T20:36:58.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2103 2024-08-25T20:36:58.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:58.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:36:58.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:58.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:58.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:36:58.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:36:58.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:36:58.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:36:58.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:36:58.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:36:58.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:36:58.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:36:58.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:58.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:59.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2106 2024-08-25T20:36:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:59.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2105 2024-08-25T20:36:59.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:59.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2109 2024-08-25T20:36:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2110 2024-08-25T20:36:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:36:59.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:00.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2108 2024-08-25T20:37:00.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2107 2024-08-25T20:37:00.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:00.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:00.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:37:00.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:00.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:00.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:37:00.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:37:00.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:01.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2112 2024-08-25T20:37:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:01.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2111 2024-08-25T20:37:01.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:01.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:37:01.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:01.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:01.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:37:01.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:37:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:37:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:37:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:37:02.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:02.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:37:02.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:02.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:02.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:37:02.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:37:02.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:02.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:37:02.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:02.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:02.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:37:02.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:37:02.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2114 2024-08-25T20:37:02.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:02.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:37:02.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:02.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:02.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:37:02.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:37:02.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:02.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:37:02.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:02.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:02.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:37:02.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:37:02.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2113 2024-08-25T20:37:02.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:02.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:37:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:37:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:37:03.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:03.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:37:03.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:03.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:03.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:37:03.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:37:03.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2116 2024-08-25T20:37:03.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:04.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2115 2024-08-25T20:37:04.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:04.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:04.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:04.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:04.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:04.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:04.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:04.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:37:04.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:04.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:04.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:37:04.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:37:04.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:04.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2120 2024-08-25T20:37:04.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2119 2024-08-25T20:37:04.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:04.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:05.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2117 2024-08-25T20:37:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2118 2024-08-25T20:37:05.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:05.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:05.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:05.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:37:05.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:05.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:05.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:37:05.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:37:05.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:05.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:37:05.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:05.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:05.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:37:05.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:37:06.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:06.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:37:06.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:06.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:06.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:37:06.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:37:07.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2121 2024-08-25T20:37:07.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2122 2024-08-25T20:37:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:07.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2124 2024-08-25T20:37:07.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:07.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2123 2024-08-25T20:37:07.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:07.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:07.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:07.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:37:07.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:07.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:07.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:37:07.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:37:07.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:07.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:37:07.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:07.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:07.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:37:07.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:37:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:37:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:07.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:37:07.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:37:08.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:08.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:37:08.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:08.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:08.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:37:08.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:37:08.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2126 2024-08-25T20:37:08.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:08.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:37:08.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:08.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:08.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:37:08.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:37:08.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:09.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2125 2024-08-25T20:37:09.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:09.713 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:37:09.713+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae3a86400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:37:09.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2129 2024-08-25T20:37:09.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2130 2024-08-25T20:37:09.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:09.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:10.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:10.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:37:10.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:10.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:10.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:37:10.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:37:10.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:10.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:10.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:10.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:10.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:10.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:10.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:10.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:37:10.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:10.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:10.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:37:10.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:37:10.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2127 2024-08-25T20:37:10.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:10.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:37:10.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:10.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:10.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:37:10.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:37:10.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2128 2024-08-25T20:37:10.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:11.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2132 2024-08-25T20:37:11.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2131 2024-08-25T20:37:11.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:11.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:11.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:11.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:37:11.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:11.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:11.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:37:11.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:37:12.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2133 2024-08-25T20:37:12.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2134 2024-08-25T20:37:12.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:12.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:12.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:12.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:37:12.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:12.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:12.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:37:12.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:37:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:37:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:37:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:37:12.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:12.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:37:12.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:12.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:12.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:37:12.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:37:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2135 2024-08-25T20:37:13.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2136 2024-08-25T20:37:13.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:13.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:37:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:13.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:37:13.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:37:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:37:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:37:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:37:14.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2138 2024-08-25T20:37:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:14.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2137 2024-08-25T20:37:14.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:37:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:37:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:37:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2139 2024-08-25T20:37:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:14.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:14.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2140 2024-08-25T20:37:14.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:14.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:37:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:37:15.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:37:15.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:15.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:37:15.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:15.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:15.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:37:15.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:37:16.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:16.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:37:16.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:16.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:16.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:37:16.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:37:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:37:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:37:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:37:17.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2141 2024-08-25T20:37:17.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2142 2024-08-25T20:37:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:17.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:17.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:37:17.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:17.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:17.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:37:17.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:37:17.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:17.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:37:17.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:17.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:17.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:37:17.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:37:17.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:17.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:17.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:37:17.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:17.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:17.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:37:17.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:37:17.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:17.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:37:17.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:17.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:17.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:37:17.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:37:18.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2143 2024-08-25T20:37:18.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2144 2024-08-25T20:37:18.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:18.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:18.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2149 2024-08-25T20:37:18.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:19.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2146 2024-08-25T20:37:19.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2145 2024-08-25T20:37:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2148 2024-08-25T20:37:19.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:19.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:19.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:20.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2147 2024-08-25T20:37:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2150 2024-08-25T20:37:20.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:20.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:20.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:20.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:20.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:20.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:20.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:37:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:37:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:37:21.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2152 2024-08-25T20:37:21.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2151 2024-08-25T20:37:21.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:21.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:21.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:21.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:37:21.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:21.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:21.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:37:21.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:37:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:37:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:37:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:37:21.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:21.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:37:21.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:21.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:21.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:37:21.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:37:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2155 2024-08-25T20:37:22.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2153 2024-08-25T20:37:22.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2154 2024-08-25T20:37:22.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:22.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:22.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:37:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:37:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:37:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:37:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:37:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:37:22.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:22.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:37:22.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:22.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:22.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:37:22.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:37:23.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:23.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:37:23.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:23.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:23.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:37:23.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:37:23.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:23.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:37:23.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:23.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:23.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:37:23.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:37:23.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2156 2024-08-25T20:37:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2157 2024-08-25T20:37:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:24.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:24.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:37:24.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:24.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:24.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:37:24.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:37:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2159 2024-08-25T20:37:24.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2160 2024-08-25T20:37:24.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:24.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:25.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2158 2024-08-25T20:37:25.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:25.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:25.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:37:25.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:25.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:25.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:37:25.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:37:25.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:25.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:37:25.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:25.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:25.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:37:25.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:37:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:37:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:37:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:37:26.564 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:37:26.563+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf99f400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:37:26.569 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:37:26.567+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae4c2c800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:37:26.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2163 2024-08-25T20:37:26.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:26.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:37:26.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:26.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:26.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:37:26.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:37:26.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:37:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:26.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:37:26.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:37:27.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2162 2024-08-25T20:37:27.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2161 2024-08-25T20:37:27.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:27.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:37:27.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:27.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:27.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:37:27.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:37:27.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:27.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:37:27.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:27.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:27.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:37:27.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:37:27.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:27.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:28.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:28.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:37:28.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:28.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:28.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:37:28.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:37:28.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2165 2024-08-25T20:37:28.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2164 2024-08-25T20:37:28.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:28.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2167 2024-08-25T20:37:29.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2166 2024-08-25T20:37:29.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:29.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:37:29.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:29.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:29.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:37:29.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:37:29.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:29.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:29.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2170 2024-08-25T20:37:29.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:30.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2169 2024-08-25T20:37:30.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2168 2024-08-25T20:37:30.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:37:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:37:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:37:30.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:30.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:30.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:30.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:30.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:30.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:31.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2172 2024-08-25T20:37:31.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2173 2024-08-25T20:37:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:31.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:31.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:37:31.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:31.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:31.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:37:31.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:37:31.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:31.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:37:31.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:31.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:31.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:37:31.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:37:32.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2171 2024-08-25T20:37:32.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:32.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2175 2024-08-25T20:37:32.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2174 2024-08-25T20:37:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:32.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:37:32.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:32.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:32.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:37:32.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:37:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:37:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:37:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:37:32.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:32.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:32.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:32.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:37:32.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:32.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:32.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:37:32.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:37:33.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:33.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:37:33.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:33.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:33.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:37:33.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:37:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:37:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:37:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:37:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2177 2024-08-25T20:37:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2176 2024-08-25T20:37:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:33.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:37:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:37:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:37:34.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2179 2024-08-25T20:37:34.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2180 2024-08-25T20:37:34.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:34.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:35.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2178 2024-08-25T20:37:35.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:35.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:37:35.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:35.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:35.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:37:35.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:37:35.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:35.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:35.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:37:35.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:35.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:35.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:37:35.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:37:35.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:35.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:37:35.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:35.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:35.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:37:35.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:37:36.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2182 2024-08-25T20:37:36.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2181 2024-08-25T20:37:36.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:36.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:36.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:36.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:37:36.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:36.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:36.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:37:36.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:37:36.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:36.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:37:36.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:36.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:36.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:37:36.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:37:37.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:37.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:37:37.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:37.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:37.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:37:37.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:37:37.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:37.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:37:37.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:37.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:37.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:37:37.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:37:38.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:38.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:37:38.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:38.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:38.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:37:38.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:37:38.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2185 2024-08-25T20:37:38.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2184 2024-08-25T20:37:38.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:38.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:39.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:39.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:37:39.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:39.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:39.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:37:39.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:37:39.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:39.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:39.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:39.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:39.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:39.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:39.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2186 2024-08-25T20:37:39.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2187 2024-08-25T20:37:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:39.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:39.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2190 2024-08-25T20:37:39.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:40.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2183 2024-08-25T20:37:40.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2189 2024-08-25T20:37:40.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2188 2024-08-25T20:37:40.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:40.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:40.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:41.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2191 2024-08-25T20:37:41.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:41.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:41.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:37:41.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:41.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:41.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:37:41.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:37:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:37:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:37:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:37:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2192 2024-08-25T20:37:42.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:42.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2193 2024-08-25T20:37:42.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2194 2024-08-25T20:37:42.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:42.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:42.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:42.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:37:42.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:42.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:42.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:37:42.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:37:42.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:42.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:37:42.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:42.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:42.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:37:42.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:37:42.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:42.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:37:42.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:42.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:42.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:37:42.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:37:43.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2196 2024-08-25T20:37:43.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2195 2024-08-25T20:37:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:43.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:43.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:37:43.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:43.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:43.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:37:43.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:37:43.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:43.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:43.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:37:43.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:43.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:43.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:37:43.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:37:43.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:43.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:37:43.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:43.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:43.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:37:43.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:37:44.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:44.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:37:44.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:44.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:44.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:37:44.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:37:44.690 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:37:44.689+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf51b800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:37:44.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2199 2024-08-25T20:37:44.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2200 2024-08-25T20:37:44.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2198 2024-08-25T20:37:44.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:44.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:44.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:45.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:45.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:37:45.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:45.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:45.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:37:45.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:45.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:37:45.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:37:45.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:45.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:45.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:37:45.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:37:45.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2197 2024-08-25T20:37:45.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:46.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2202 2024-08-25T20:37:46.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:37:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:37:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:37:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:37:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:37:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:37:47.543 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2201 2024-08-25T20:37:47.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:37:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:37:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:37:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:37:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:47.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:47.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:37:47.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:37:47.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:47.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:37:47.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:47.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:47.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:37:47.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:37:48.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2203 2024-08-25T20:37:48.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2204 2024-08-25T20:37:48.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:48.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:48.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:48.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:37:48.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:48.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:48.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:37:48.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:37:49.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2205 2024-08-25T20:37:49.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2206 2024-08-25T20:37:49.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:49.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:49.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:49.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:49.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:49.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:49.592 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:37:49.592+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ade586c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:37:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:49.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:49.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2210 2024-08-25T20:37:49.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:50.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2209 2024-08-25T20:37:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2207 2024-08-25T20:37:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2208 2024-08-25T20:37:50.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:50.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:50.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:50.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:37:50.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:50.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:50.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:37:50.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:37:50.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:51.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2212 2024-08-25T20:37:51.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:51.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:51.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:37:51.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:51.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:51.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:37:51.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:51.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:37:51.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:37:52.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2211 2024-08-25T20:37:52.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:52.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2213 2024-08-25T20:37:52.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2214 2024-08-25T20:37:52.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:52.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:37:52.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:52.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:52.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:37:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:37:52.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:52.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:52.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:37:52.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:52.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:52.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:37:52.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:37:52.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:52.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:37:52.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:52.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:52.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:37:52.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:37:53.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:53.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:37:53.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:53.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:53.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:37:53.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:37:53.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2216 2024-08-25T20:37:53.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:53.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2215 2024-08-25T20:37:53.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:37:53.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:53.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:37:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:37:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2217 2024-08-25T20:37:53.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:53.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:37:53.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:37:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:37:53.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:53.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:53.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:37:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:37:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:37:54.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2220 2024-08-25T20:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2219 2024-08-25T20:37:54.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2218 2024-08-25T20:37:54.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:54.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:55.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:55.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:55.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:55.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:55.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:37:55.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:55.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:55.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:37:55.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:37:55.609 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:37:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:37:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:37:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2222 2024-08-25T20:37:56.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:56.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:56.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:37:56.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:56.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:56.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:37:56.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:37:56.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:56.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:37:56.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:56.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:56.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:37:56.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:37:56.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:56.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:37:56.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:56.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:56.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:37:56.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:37:57.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2221 2024-08-25T20:37:57.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:57.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:37:57.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:57.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:57.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:37:57.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:37:57.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:57.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:37:57.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:57.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:57.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:37:57.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:37:57.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:57.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:37:57.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:57.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:57.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:37:57.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:37:57.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:57.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2227 2024-08-25T20:37:57.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:58.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2224 2024-08-25T20:37:58.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2223 2024-08-25T20:37:58.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:58.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:58.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2228 2024-08-25T20:37:58.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2225 2024-08-25T20:37:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:37:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:37:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:37:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:37:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:37:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:37:59.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2226 2024-08-25T20:37:59.629 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:37:59.628+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf857c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:37:59.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:37:59.641 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:37:59.641+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae39bac00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:37:59.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:00.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2230 2024-08-25T20:38:00.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2229 2024-08-25T20:38:00.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:00.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:00.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:00.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:38:00.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:00.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:00.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:38:00.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:38:00.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:00.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:38:00.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:00.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:00.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:38:00.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:38:01.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2232 2024-08-25T20:38:01.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:01.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:01.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:38:01.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:01.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:01.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:38:01.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:38:01.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:01.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:38:01.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:01.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:01.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:38:01.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:38:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:38:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:38:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:38:02.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2231 2024-08-25T20:38:02.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:02.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2235 2024-08-25T20:38:02.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2234 2024-08-25T20:38:02.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:02.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:02.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:02.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:38:02.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:02.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:02.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:38:02.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:38:02.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:02.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:38:02.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:02.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:02.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:38:02.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:38:03.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:03.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:38:03.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:03.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:03.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:38:03.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:38:03.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:03.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:38:03.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:03.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:03.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:38:03.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:38:03.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2238 2024-08-25T20:38:03.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2237 2024-08-25T20:38:03.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:03.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:38:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:38:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:38:04.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2233 2024-08-25T20:38:04.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:04.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2240 2024-08-25T20:38:04.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2239 2024-08-25T20:38:04.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:04.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:05.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:05.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:38:05.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:05.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:05.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:38:05.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:38:05.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2236 2024-08-25T20:38:05.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:05.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:38:05.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:05.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:05.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:38:05.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:38:05.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:05.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:38:05.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:05.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:05.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:38:05.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:38:05.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:06.566 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:38:06.566+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf77a800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:38:06.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2242 2024-08-25T20:38:06.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:06.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:06.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:38:06.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:06.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:06.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:38:06.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:38:06.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:06.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:38:06.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:06.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:06.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:38:06.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:38:07.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2241 2024-08-25T20:38:07.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:07.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:07.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:38:07.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:07.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:07.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:38:07.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:38:07.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:07.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:38:07.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:07.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:07.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:38:07.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:38:07.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:07.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:38:07.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:07.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:07.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:38:07.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:38:08.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2243 2024-08-25T20:38:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:38:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:38:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:38:08.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2244 2024-08-25T20:38:08.660 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:38:08.659+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ade27cc00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:38:08.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:08.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:08.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2248 2024-08-25T20:38:08.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2249 2024-08-25T20:38:08.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:08.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:09.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:09.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:38:09.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:09.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:09.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:38:09.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:38:09.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2245 2024-08-25T20:38:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2246 2024-08-25T20:38:09.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:09.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:09.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2250 2024-08-25T20:38:09.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:10.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:10.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:38:10.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:10.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:10.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:38:10.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:38:10.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2247 2024-08-25T20:38:10.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:11.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2251 2024-08-25T20:38:11.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:11.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2252 2024-08-25T20:38:11.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:11.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:11.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:38:11.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:11.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:11.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:38:11.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:38:11.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:11.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:38:11.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:11.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:11.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:38:11.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:38:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:38:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:38:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:38:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:38:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:38:11.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:38:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2256 2024-08-25T20:38:12.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2254 2024-08-25T20:38:12.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2255 2024-08-25T20:38:12.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2253 2024-08-25T20:38:12.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:12.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:38:12.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:12.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:12.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:38:12.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:38:12.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:12.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:38:12.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:12.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:12.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:38:12.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:38:12.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:12.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:12.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:12.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:12.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:12.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:38:12.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:12.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:12.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:38:12.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:38:13.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:13.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:38:13.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:13.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:13.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:38:13.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:38:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2257 2024-08-25T20:38:13.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2259 2024-08-25T20:38:13.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:14.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:14.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:38:14.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:14.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:14.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:38:14.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:38:14.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:38:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:38:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:38:14.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2260 2024-08-25T20:38:14.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:15.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2258 2024-08-25T20:38:15.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:15.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2262 2024-08-25T20:38:15.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:15.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:38:15.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:15.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:15.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:15.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:38:15.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:38:15.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:38:15.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:15.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:15.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:38:15.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:38:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:38:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:38:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:38:15.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:15.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:38:15.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:15.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:15.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:38:15.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:38:15.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:16.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2263 2024-08-25T20:38:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2265 2024-08-25T20:38:16.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:16.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:16.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:16.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:38:16.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:16.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:16.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:38:16.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:38:16.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:16.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:38:16.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:16.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:16.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:38:16.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:38:17.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2261 2024-08-25T20:38:17.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:17.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2268 2024-08-25T20:38:17.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:17.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:17.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:38:17.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:17.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:17.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:38:17.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:38:18.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2266 2024-08-25T20:38:18.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2264 2024-08-25T20:38:18.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:18.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:18.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:18.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:38:18.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:18.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:18.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:38:18.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:38:18.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:18.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:38:18.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:18.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:18.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:38:18.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:38:19.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2267 2024-08-25T20:38:19.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2270 2024-08-25T20:38:19.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2271 2024-08-25T20:38:19.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:19.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:38:19.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:19.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:19.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:38:19.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:38:19.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:19.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:38:19.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:19.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:19.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:38:19.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:38:19.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:19.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:20.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2269 2024-08-25T20:38:20.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:20.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:20.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:38:20.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:20.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:20.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:38:20.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:38:20.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:20.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:38:20.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:20.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:20.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:38:20.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:38:21.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2272 2024-08-25T20:38:21.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2273 2024-08-25T20:38:21.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:38:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:38:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:38:21.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:21.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:38:21.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:21.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:21.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:38:21.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:38:21.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2275 2024-08-25T20:38:21.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2274 2024-08-25T20:38:21.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:21.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2276 2024-08-25T20:38:22.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2277 2024-08-25T20:38:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:38:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:38:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:38:22.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:22.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:22.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:38:22.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:22.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:22.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:38:22.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:38:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:38:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:38:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:38:23.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:23.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:38:23.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:23.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:23.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:38:23.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:38:23.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2278 2024-08-25T20:38:23.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2279 2024-08-25T20:38:23.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:23.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:38:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:38:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:38:24.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2281 2024-08-25T20:38:24.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:24.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:38:24.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:24.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:24.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:38:24.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:38:24.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:24.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:24.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:38:24.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:24.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:24.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:38:24.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:38:24.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:24.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:38:24.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:24.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:24.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:38:24.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:38:25.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2280 2024-08-25T20:38:25.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:25.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:38:25.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:25.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:25.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:38:25.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:38:25.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:25.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:38:25.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:25.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:25.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:38:25.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:38:25.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:25.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2285 2024-08-25T20:38:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:26.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2283 2024-08-25T20:38:26.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2282 2024-08-25T20:38:26.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:26.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:38:26.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:26.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:26.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:38:26.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:38:26.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:26.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:38:26.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:26.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:26.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:38:26.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:38:26.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:26.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:27.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2284 2024-08-25T20:38:27.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2286 2024-08-25T20:38:28.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2287 2024-08-25T20:38:28.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2288 2024-08-25T20:38:28.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:38:28.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 94967 2024-08-25T20:38:28.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:38:28.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 95099 2024-08-25T20:38:28.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:28.626 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:38:28.623+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae5042c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:38:28.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:28.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:29.610 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2289 2024-08-25T20:38:29.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:29.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:38:29.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:29.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:29.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:38:29.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:38:29.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:30.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:38:30.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 95231 2024-08-25T20:38:31.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:38:31.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 95383 2024-08-25T20:38:31.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:38:31.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 95536 2024-08-25T20:38:31.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:38:31.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 95710 2024-08-25T20:38:31.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:31.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:38:31.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:31.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:31.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:38:31.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:38:31.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:38:31.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 95883 2024-08-25T20:38:32.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2290 2024-08-25T20:38:32.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:32.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2291 2024-08-25T20:38:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:32.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:38:32.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:38:32.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:38:32.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:38:32.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:38:32.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:38:33.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2292 2024-08-25T20:38:33.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:38:35.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:38:35.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 96057 2024-08-25T20:38:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:38:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 96211 2024-08-25T20:38:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:38:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:38:36.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 94836 2024-08-25T20:41:03.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:41:03.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:41:03.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:41:03.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 94968 2024-08-25T20:41:04.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:41:04.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:41:04.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:41:04.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 95100 2024-08-25T20:41:05.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:41:05.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:41:05.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:41:05.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 95232 2024-08-25T20:41:06.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:41:06.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:41:06.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:41:06.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 95384 2024-08-25T20:41:07.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:41:07.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:41:07.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:41:07.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 95537 2024-08-25T20:41:08.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:41:08.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:41:08.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:41:08.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 95711 2024-08-25T20:41:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:41:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:41:10.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:41:10.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 95884 2024-08-25T20:41:11.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:41:11.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:41:11.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:41:11.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 96058 2024-08-25T20:41:12.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:41:12.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:41:12.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:41:12.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 96212 2024-08-25T20:41:13.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:41:13.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:41:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:41:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:41:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:41:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:41:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 1 2024-08-25T20:41:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd0 2024-08-25T20:41:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary1 2024-08-25T20:41:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary1 2024-08-25T20:41:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:41:13.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary1 2024-08-25T20:41:13.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:41:13.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:41:13.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 2 2024-08-25T20:41:13.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd1 2024-08-25T20:41:13.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary2 2024-08-25T20:41:13.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary2 2024-08-25T20:41:13.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:41:13.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary2 2024-08-25T20:41:13.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:41:13.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:41:13.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 3 2024-08-25T20:41:13.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd2 2024-08-25T20:41:13.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary3 2024-08-25T20:41:13.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary3 2024-08-25T20:41:13.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:41:13.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary3 2024-08-25T20:41:13.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:41:13.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:41:13.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 4 2024-08-25T20:41:13.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd3 2024-08-25T20:41:13.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary4 2024-08-25T20:41:13.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary4 2024-08-25T20:41:13.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:41:13.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary4 2024-08-25T20:41:13.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:41:13.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:41:13.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 5 2024-08-25T20:41:13.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:41:13.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd4 2024-08-25T20:41:13.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary5 2024-08-25T20:41:13.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary5 2024-08-25T20:41:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:41:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:41:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary5 2024-08-25T20:41:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 6 2024-08-25T20:41:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd5 2024-08-25T20:41:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary6 2024-08-25T20:41:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary6 2024-08-25T20:41:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:41:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary6 2024-08-25T20:41:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:41:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 7 2024-08-25T20:41:13.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:41:13.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd6 2024-08-25T20:41:13.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary7 2024-08-25T20:41:13.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary7 2024-08-25T20:41:13.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:41:13.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary7 2024-08-25T20:41:13.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:41:13.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:41:13.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:41:13.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:41:13.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:41:13.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:41:13.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 10 2024-08-25T20:41:13.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 102166 2024-08-25T20:41:13.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd9 2024-08-25T20:41:13.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 8 2024-08-25T20:41:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd7 2024-08-25T20:41:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 9 2024-08-25T20:41:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary8 2024-08-25T20:41:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd8 2024-08-25T20:41:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary8 2024-08-25T20:41:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:41:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary8 2024-08-25T20:41:13.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary9 2024-08-25T20:41:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary9 2024-08-25T20:41:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary10 2024-08-25T20:41:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary10 2024-08-25T20:41:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:41:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary10 2024-08-25T20:41:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:41:13.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary9 2024-08-25T20:41:14.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:41:14.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:41:14.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:41:14.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:41:14.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:41:14.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:41:14.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:41:14.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:41:14.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:41:14.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:41:14.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:41:14.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:41:14.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:41:14.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:41:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:41:14.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:41:14.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:41:14.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:41:14.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:41:14.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:42:08.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=52471b9404c85f659911705ff902143c 2024-08-25T20:42:08.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-08-25T20:42:08.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=b996c713be26eb2e3ae5400e70062551 2024-08-25T20:42:08.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-08-25T20:42:09.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary9 2024-08-25T20:42:09.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:09.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:09.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:42:09.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-08-25T20:42:09.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=db7cde1e00ab7a765d88eafee2e7d8aa 2024-08-25T20:42:09.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-08-25T20:42:09.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary3 2024-08-25T20:42:09.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:09.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:09.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:42:09.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-08-25T20:42:09.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary6 2024-08-25T20:42:09.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:09.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:09.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:42:09.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-08-25T20:42:09.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=7fbbfe32b9605446f48a239b1b8c5c05 2024-08-25T20:42:09.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-08-25T20:42:09.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=effc958b150ce8355bbb2c0d3ef539b7 2024-08-25T20:42:09.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-08-25T20:42:09.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary5 2024-08-25T20:42:09.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:09.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:09.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:42:09.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-08-25T20:42:09.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=a48c78bcf94b7f4150eb518e716232c9 2024-08-25T20:42:09.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-08-25T20:42:09.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary8 2024-08-25T20:42:09.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:09.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:09.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:42:09.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-08-25T20:42:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=e52a8be352b807af7d8c0938dbb38453 2024-08-25T20:42:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-08-25T20:42:09.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary4 2024-08-25T20:42:09.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:09.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:09.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:42:09.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-08-25T20:42:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=d79c8dfd929175a28d7fff38b895d028 2024-08-25T20:42:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-08-25T20:42:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary1 2024-08-25T20:42:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:42:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-08-25T20:42:09.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:42:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-08-25T20:42:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:42:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:10.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary7 2024-08-25T20:42:10.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:10.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:10.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:42:10.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-08-25T20:42:10.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=0f35411bf327e153ff073fc7456a30c5 2024-08-25T20:42:10.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-08-25T20:42:10.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=1f01bbd50ae44e4591fbb57c893e4c47 2024-08-25T20:42:10.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-08-25T20:42:10.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary2 2024-08-25T20:42:10.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:10.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:10.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:42:10.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-08-25T20:42:10.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary10 2024-08-25T20:42:10.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:10.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:10.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:42:10.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-08-25T20:42:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:42:11.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:42:11.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:11.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:11.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:42:11.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:11.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:11.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:11.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:11.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:11.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:42:11.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:42:11.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-08-25T20:42:11.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:11.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:11.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:42:11.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:11.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:11.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:11.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:11.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:11.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:42:11.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:42:11.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-08-25T20:42:11.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:11.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:42:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:11.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-08-25T20:42:11.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:11.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:11.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:42:11.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:11.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:11.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:11.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:11.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:11.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-08-25T20:42:11.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:11.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:11.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:42:11.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:11.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:11.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:11.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:11.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:11.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:42:11.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 629954bc-57dc-455c-84fe-8764b2a92c2e 2024-08-25T20:42:11.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:11.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:11.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:11.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:11.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:11.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:11.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:11.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:42:12.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:42:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-08-25T20:42:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:42:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:12.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:12.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:42:12.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:12.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:12.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:42:12.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:12.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:12.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:12.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:12.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:12.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:42:12.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:42:12.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:12.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:12.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:42:12.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:12.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:12.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:12.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:42:12.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:42:12.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:12.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:12.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:42:12.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:12.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:12.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:12.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:12.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:12.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:42:12.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:42:12.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:12.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:12.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:42:12.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:12.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:12.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:12.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:12.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:12.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:42:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:42:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 72b47598-0266-4d1c-81e0-f751f633b29c 2024-08-25T20:42:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:12.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:12.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:12.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:42:12.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:42:12.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8d09bd3d-e631-4259-85a7-7df19d5d05a5 2024-08-25T20:42:12.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:12.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:12.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:12.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:12.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:12.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:12.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:12.315 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:42:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:42:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3d225f7-8cf1-4273-85ce-9973dfd95e9a 2024-08-25T20:42:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:12.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:12.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:12.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:42:12.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:42:12.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d8ea85d6-0a14-489d-88c9-253afe4f3a5c 2024-08-25T20:42:12.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:12.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:12.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:12.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:12.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:12.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:12.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:12.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:42:13.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:42:13.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:13.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:13.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:42:13.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:13.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:13.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:13.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:13.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:13.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:42:13.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:42:13.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ae736451-33b4-4280-b078-499058ffab98 2024-08-25T20:42:13.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:13.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:13.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:13.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:13.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:13.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:13.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:13.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:42:13.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:42:13.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:13.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:13.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:42:13.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:13.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:13.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:13.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:13.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:13.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:42:13.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:42:13.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 629954bc-57dc-455c-84fe-8764b2a92c2e 2024-08-25T20:42:13.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:13.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:13.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:13.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:13.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:13.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:13.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:13.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:42:14.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:42:14.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:42:14.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:42:14.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-08-25T20:42:14.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:14.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:14.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:42:14.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:14.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:14.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:14.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:14.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:14.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-08-25T20:42:14.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:14.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:14.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:42:14.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:14.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:14.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:14.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:14.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-08-25T20:42:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:42:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:42:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:42:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:14.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:14.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:42:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:42:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:42:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:14.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:14.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:42:14.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:42:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:42:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:14.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:14.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:42:14.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:42:14.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 72b47598-0266-4d1c-81e0-f751f633b29c 2024-08-25T20:42:14.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:14.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:14.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:14.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:14.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:14.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:14.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:14.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:42:14.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:42:14.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:14.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:14.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:42:14.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:14.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:14.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:14.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:14.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:14.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:42:14.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:42:14.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8d09bd3d-e631-4259-85a7-7df19d5d05a5 2024-08-25T20:42:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:42:14.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:42:14.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3d225f7-8cf1-4273-85ce-9973dfd95e9a 2024-08-25T20:42:14.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:14.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:14.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:14.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:14.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:14.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:14.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:14.449 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:42:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:42:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d8ea85d6-0a14-489d-88c9-253afe4f3a5c 2024-08-25T20:42:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:14.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:14.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:14.458 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:42:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:42:15.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-08-25T20:42:15.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:15.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:15.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:42:15.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:15.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:15.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:15.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:15.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:15.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:42:15.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:15.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:15.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:42:15.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:15.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:15.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:15.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:15.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:15.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:42:15.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:42:15.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:15.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:15.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:42:15.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:15.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:15.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:15.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:15.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:15.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:42:15.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:42:15.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:15.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:15.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:42:15.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:15.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:15.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:15.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:15.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:15.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:42:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:42:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8ea7a427-cfbf-4f64-b72c-363c5306e09f 2024-08-25T20:42:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:15.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:42:15.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:42:15.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7a92202a-68ae-4d01-928a-945468468e5c 2024-08-25T20:42:15.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:15.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:15.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:15.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:15.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:15.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:42:15.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1d9ccc9-fc58-49ab-9a15-08aef159ef5f 2024-08-25T20:42:15.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:15.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:15.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:15.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:15.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:15.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:15.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:15.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:42:15.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:15.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:15.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:42:15.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:42:15.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:15.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:15.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:42:15.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:15.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:15.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:15.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:15.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:15.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:42:15.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:42:15.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ae736451-33b4-4280-b078-499058ffab98 2024-08-25T20:42:15.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:15.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:15.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:15.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:15.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:15.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:15.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:15.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:42:16.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:42:16.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:16.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:16.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:42:16.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:16.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:16.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:16.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:42:16.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:42:16.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7da99107-e1ee-4201-a999-13a8eec52934 2024-08-25T20:42:16.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:16.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:16.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:16.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:16.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:16.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:16.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:16.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:42:17.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:42:17.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:17.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:17.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:42:17.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:17.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:17.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:17.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:17.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:17.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:42:17.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:17.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:42:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:42:17.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:42:17.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8ea7a427-cfbf-4f64-b72c-363c5306e09f 2024-08-25T20:42:17.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:17.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:17.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:17.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:17.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:17.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:42:17.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1d9ccc9-fc58-49ab-9a15-08aef159ef5f 2024-08-25T20:42:17.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:17.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:17.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:17.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:17.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:17.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:17.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:17.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:42:17.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:42:17.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7a92202a-68ae-4d01-928a-945468468e5c 2024-08-25T20:42:17.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:17.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:17.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:17.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:17.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:17.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:17.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:17.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:42:17.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:17.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:17.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:42:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:42:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:42:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:42:17.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:42:17.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 629954bc-57dc-455c-84fe-8764b2a92c2e 2024-08-25T20:42:17.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:17.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:17.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:17.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:17.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:17.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:42:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:42:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:18.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:18.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:42:18.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:42:18.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7da99107-e1ee-4201-a999-13a8eec52934 2024-08-25T20:42:18.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:18.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:18.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:18.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:18.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:18.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:18.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:18.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:42:18.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:42:18.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:18.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:18.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:42:18.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:18.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:18.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:18.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:18.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:18.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:42:18.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:42:18.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:18.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:18.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:42:18.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:18.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:18.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:18.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:18.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:18.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:42:18.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:42:18.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:18.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:18.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:42:18.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:18.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:18.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:18.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:18.452 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:18.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:42:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:42:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:42:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:42:18.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:42:18.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 72b47598-0266-4d1c-81e0-f751f633b29c 2024-08-25T20:42:18.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:18.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:18.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:18.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:18.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:18.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:42:18.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8d09bd3d-e631-4259-85a7-7df19d5d05a5 2024-08-25T20:42:18.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:18.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:18.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:18.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:18.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:18.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:18.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:18.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:18.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:42:18.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3d225f7-8cf1-4273-85ce-9973dfd95e9a 2024-08-25T20:42:18.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:18.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:18.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:18.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:18.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:18.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:42:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d8ea85d6-0a14-489d-88c9-253afe4f3a5c 2024-08-25T20:42:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:18.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:18.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:18.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:19.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:42:19.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:19.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:19.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:42:19.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:19.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:19.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:19.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:19.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:19.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:42:19.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:42:19.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ae736451-33b4-4280-b078-499058ffab98 2024-08-25T20:42:19.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:19.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:19.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:19.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:19.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:19.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:19.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:19.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:21.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:42:21.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:21.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:21.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:42:21.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:42:21.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:42:21.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:21.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:21.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:42:21.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:21.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:21.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:21.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:42:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:42:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:21.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:42:21.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:21.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:42:21.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:42:21.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1d9ccc9-fc58-49ab-9a15-08aef159ef5f 2024-08-25T20:42:21.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:21.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:21.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:21.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:21.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:21.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:21.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:21.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:21.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:42:21.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7a92202a-68ae-4d01-928a-945468468e5c 2024-08-25T20:42:21.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:21.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:21.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:21.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:21.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:42:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8ea7a427-cfbf-4f64-b72c-363c5306e09f 2024-08-25T20:42:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:42:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:42:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:22.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:22.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:22.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:22.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:42:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:42:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7da99107-e1ee-4201-a999-13a8eec52934 2024-08-25T20:42:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:22.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:22.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:22.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:42:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:42:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:25.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:25.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:42:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:42:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 629954bc-57dc-455c-84fe-8764b2a92c2e 2024-08-25T20:42:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:25.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:25.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:25.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:26.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:42:26.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:26.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:26.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:42:26.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:26.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:26.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:26.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:26.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:26.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:42:26.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:42:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:42:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:26.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:42:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:42:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:42:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:26.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:26.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:42:26.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:42:26.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:26.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:26.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:42:26.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:26.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:26.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:26.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:26.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:26.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:42:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:42:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 72b47598-0266-4d1c-81e0-f751f633b29c 2024-08-25T20:42:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:26.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:26.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:26.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:26.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:42:26.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8d09bd3d-e631-4259-85a7-7df19d5d05a5 2024-08-25T20:42:26.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:26.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:26.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:26.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:26.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:26.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:26.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:26.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:26.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:42:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3d225f7-8cf1-4273-85ce-9973dfd95e9a 2024-08-25T20:42:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:42:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d8ea85d6-0a14-489d-88c9-253afe4f3a5c 2024-08-25T20:42:26.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:26.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:26.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:26.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:26.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:26.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:26.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:26.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:26.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:26.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:26.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:27.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:42:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:42:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:42:27.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:42:27.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ae736451-33b4-4280-b078-499058ffab98 2024-08-25T20:42:27.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:42:27.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:42:27.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:27.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:41:58' 2024-08-25T20:42:27.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:27.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:42:27.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:27.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:42:29.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:42:29.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:29.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:29.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:42:29.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:29.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:29.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:29.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:29.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:29.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:42:29.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:42:29.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:29.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:29.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:29.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:29.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:42:29.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:29.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:42:29.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:42:29.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1d9ccc9-fc58-49ab-9a15-08aef159ef5f 2024-08-25T20:42:29.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:29.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:29.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:29.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:29.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:29.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:42:29.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7a92202a-68ae-4d01-928a-945468468e5c 2024-08-25T20:42:29.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:29.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:29.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:29.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:29.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:29.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:29.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:29.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:29.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:42:29.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8ea7a427-cfbf-4f64-b72c-363c5306e09f 2024-08-25T20:42:29.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:29.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:29.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:29.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:29.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:29.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:29.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:29.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:29.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:29.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:29.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:29.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:29.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:29.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:29.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:29.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:29.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:29.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:29.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:29.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:29.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:29.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:29.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:29.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:29.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:29.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:29.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:29.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:29.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:29.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:29.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:29.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:29.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:29.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:29.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:29.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:29.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:29.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:29.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:29.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:29.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:29.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:29.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:29.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:29.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:29.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:42:29.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:29.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:29.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:29.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:29.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:29.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:29.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:29.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:42:29.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:29.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:29.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:42:29.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:29.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:29.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:42:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:42:29.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:42:29.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:42:29.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:29.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:29.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:29.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:29.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:29.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:29.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:29.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:29.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:29.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:29.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:29.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-08-25T20:42:29.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:30.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:42:30.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:30.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:30.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:42:30.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:30.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:30.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:30.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:30.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:30.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:42:30.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:42:30.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7da99107-e1ee-4201-a999-13a8eec52934 2024-08-25T20:42:30.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:30.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:30.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:30.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:30.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:30.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:30.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:30.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:30.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:30.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:30.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:30.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:30.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:30.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:30.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:30.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:30.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:30.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:30.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:30.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:30.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:42:30.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:30.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:30.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:30.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:30.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:30.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:30.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:30.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:42:30.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:30.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:42:30.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:30.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:30.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:30.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:30.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:30.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:30.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-08-25T20:42:30.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:30.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:30.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:42:30.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:30.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:30.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:30.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:30.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:30.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-08-25T20:42:30.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:30.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:30.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:42:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-08-25T20:42:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-08-25T20:42:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-08-25T20:42:30.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:42:30.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8ea7a427-cfbf-4f64-b72c-363c5306e09f 2024-08-25T20:42:30.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:30.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:30.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:30.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:30.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:42:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7a92202a-68ae-4d01-928a-945468468e5c 2024-08-25T20:42:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:42:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1d9ccc9-fc58-49ab-9a15-08aef159ef5f 2024-08-25T20:42:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:30.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:30.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:30.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:30.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:30.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:30.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:30.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:30.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:30.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:30.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:30.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:30.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:30.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:30.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:30.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:30.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:30.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:30.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:30.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:30.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:30.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:30.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:30.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:30.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:30.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:30.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:30.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:30.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:42:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:30.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:30.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:30.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:30.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:42:30.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:42:30.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:30.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:30.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:42:30.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:42:30.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:30.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:30.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:42:30.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:30.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:30.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:30.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:30.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:30.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:30.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:30.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:30.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:42:30.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:30.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:30.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary5 2024-08-25T20:42:30.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:30.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:30.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:42:30.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:42:30.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-08-25T20:42:30.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary10 2024-08-25T20:42:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:42:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:42:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-08-25T20:42:30.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:30.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:30.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:30.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:30.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary6 2024-08-25T20:42:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:42:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:42:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-08-25T20:42:31.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:42:31.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:42:31.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:42:31.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:42:31.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:42:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-08-25T20:42:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:42:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:42:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:42:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:42:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-08-25T20:42:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-08-25T20:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-08-25T20:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-08-25T20:42:31.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd1 2024-08-25T20:42:31.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:42:31.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd0 2024-08-25T20:42:31.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd2 2024-08-25T20:42:31.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:42:31.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:42:31.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:42:31.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:42:31.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7da99107-e1ee-4201-a999-13a8eec52934 2024-08-25T20:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:31.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:31.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:31.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:31.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:31.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:31.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:31.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:31.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:31.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:31.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:31.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:31.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:31.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:31.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:42:31.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:31.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:31.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:31.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:31.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:31.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:31.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:42:31.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:31.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:42:31.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:31.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:31.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:31.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:31.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:31.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:31.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary7 2024-08-25T20:42:31.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:31.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:31.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:42:31.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:42:31.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-08-25T20:42:32.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:42:32.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:42:32.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:42:32.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-08-25T20:42:32.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd3 2024-08-25T20:42:32.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:42:32.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:42:33.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:42:33.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:33.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:33.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:42:33.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:33.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:33.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:33.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:33.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:33.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:42:33.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:42:33.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 629954bc-57dc-455c-84fe-8764b2a92c2e 2024-08-25T20:42:33.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:33.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:33.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:33.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:33.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:33.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:33.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:33.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:33.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:33.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:33.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:33.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:33.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:33.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:33.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:33.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:42:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:33.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:33.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:33.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:42:33.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:42:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:33.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:33.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-08-25T20:42:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:42:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:34.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:42:34.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:34.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:34.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:42:34.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:34.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:34.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:34.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:34.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:34.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:42:34.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:42:34.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:34.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:34.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:42:34.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:34.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:34.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:34.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:34.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:34.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:42:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:42:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:42:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:34.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:42:34.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:34.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:34.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:42:34.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:34.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:34.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:34.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:34.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:34.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:42:34.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:34.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:42:34.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:42:34.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 72b47598-0266-4d1c-81e0-f751f633b29c 2024-08-25T20:42:34.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:34.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:34.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:34.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:34.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:34.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:34.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:34.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:42:34.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8d09bd3d-e631-4259-85a7-7df19d5d05a5 2024-08-25T20:42:34.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:34.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:34.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:34.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:34.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:34.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:34.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:34.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:34.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:34.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:34.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:34.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:34.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:34.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:34.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:34.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:34.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:34.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:42:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:34.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:34.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:34.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:34.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:34.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:34.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:34.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:34.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:42:34.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:34.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:34.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:34.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:34.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:42:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d8ea85d6-0a14-489d-88c9-253afe4f3a5c 2024-08-25T20:42:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:42:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3d225f7-8cf1-4273-85ce-9973dfd95e9a 2024-08-25T20:42:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:34.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:34.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:34.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:34.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:34.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:34.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:34.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:34.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:34.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:34.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:34.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:34.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:34.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:34.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:34.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:34.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:34.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:34.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:34.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:34.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:42:34.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:34.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:34.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:34.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:34.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:42:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:34.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:34.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:34.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-08-25T20:42:34.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:34.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:34.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:42:34.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:34.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:34.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:34.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:34.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:34.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-08-25T20:42:34.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:34.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:34.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:42:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:34.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:42:34.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:34.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:34.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:34.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:34.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:34.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:42:34.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:34.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:34.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:34.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-08-25T20:42:34.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:34.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:34.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:42:34.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:34.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:34.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:34.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:34.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:34.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:34.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:42:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:34.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:34.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:34.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:34.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-08-25T20:42:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:42:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:34.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:42:34.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 629954bc-57dc-455c-84fe-8764b2a92c2e 2024-08-25T20:42:34.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:34.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:34.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:34.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:34.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:34.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:34.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:34.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:34.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:34.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:34.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:34.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:42:34.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:34.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:34.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:42:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:34.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:42:35.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:35.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:35.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:35.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:35.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:42:35.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:35.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:35.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:35.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:35.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:35.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:35.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:35.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:35.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:35.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-08-25T20:42:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:42:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:35.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:35.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:35.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:35.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:35.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:42:35.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:35.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-08-25T20:42:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:42:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:35.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:35.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:35.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:42:35.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:35.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:42:35.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:35.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:35.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:35.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:35.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:35.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:35.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary3 2024-08-25T20:42:35.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:35.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:35.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:42:35.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:42:35.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-08-25T20:42:35.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:42:35.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:35.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:35.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:42:35.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:35.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:35.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:35.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:35.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:35.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:42:35.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:42:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:42:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ae736451-33b4-4280-b078-499058ffab98 2024-08-25T20:42:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:35.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:35.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:35.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:35.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:35.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:35.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:35.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:35.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:42:35.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:42:35.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-08-25T20:42:35.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:35.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:35.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:35.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:42:35.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:35.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:42:35.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:35.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:35.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:35.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:35.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:35.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:35.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:35.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:42:35.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:35.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:42:35.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:35.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:35.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:35.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:35.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd4 2024-08-25T20:42:35.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:35.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:35.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-08-25T20:42:35.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:35.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:35.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:42:35.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:35.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:35.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:35.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:42:35.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:42:35.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:42:35.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:42:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-08-25T20:42:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:42:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-08-25T20:42:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-08-25T20:42:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:42:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:35.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:35.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:35.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-08-25T20:42:36.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-08-25T20:42:36.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:36.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:36.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:42:36.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:36.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:36.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:36.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:36.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:36.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-08-25T20:42:36.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-08-25T20:42:36.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:36.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:36.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:42:36.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:36.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:36.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:36.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:36.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:36.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-08-25T20:42:36.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:42:36.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8d09bd3d-e631-4259-85a7-7df19d5d05a5 2024-08-25T20:42:36.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:36.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:36.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:36.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:36.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:36.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:36.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:36.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:36.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:42:36.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 72b47598-0266-4d1c-81e0-f751f633b29c 2024-08-25T20:42:36.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:36.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:36.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:36.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:36.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:36.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:36.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:36.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:36.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:36.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:36.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:36.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:36.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:36.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:36.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:36.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:36.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:36.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:36.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:36.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:36.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:36.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:36.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:36.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:36.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:42:36.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:36.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:36.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:36.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:36.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:36.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:36.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:36.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:36.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:36.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:36.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:36.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:36.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:36.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:42:36.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:36.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:36.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:36.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:36.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:36.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:42:36.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3d225f7-8cf1-4273-85ce-9973dfd95e9a 2024-08-25T20:42:36.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:36.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:36.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:36.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:36.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:36.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:36.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:36.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:36.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:36.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:36.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:36.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:36.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:36.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:36.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:36.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:42:36.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d8ea85d6-0a14-489d-88c9-253afe4f3a5c 2024-08-25T20:42:36.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:36.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:36.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:36.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:36.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:36.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:36.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:36.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:36.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:36.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:42:36.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:36.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:36.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:36.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:36.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:36.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:36.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:36.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:36.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:36.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:36.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:36.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:36.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:36.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:36.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:36.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:36.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:36.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:36.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:36.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:42:36.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:36.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:36.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:36.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:36.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:36.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:36.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:36.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:42:36.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:36.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:42:36.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:36.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:36.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:42:36.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:36.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:36.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:36.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:36.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:42:36.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:36.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:36.519 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:36.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary8 2024-08-25T20:42:36.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:36.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:36.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:42:36.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:42:36.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-08-25T20:42:36.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:36.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:36.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:36.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:36.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:36.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:42:36.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:36.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:36.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:36.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary1 2024-08-25T20:42:36.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:36.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:36.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:42:36.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:42:36.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-08-25T20:42:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:36.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:42:36.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:36.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:36.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:36.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:36.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:36.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:36.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary9 2024-08-25T20:42:36.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:36.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:42:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:42:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-08-25T20:42:36.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:36.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:36.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:42:36.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:36.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:42:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:36.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:36.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:36.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:36.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:36.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary4 2024-08-25T20:42:36.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:36.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:36.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:42:36.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:42:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-08-25T20:42:36.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-08-25T20:42:36.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:36.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:36.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:42:36.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:36.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:42:36.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:42:36.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:36.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:36.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-08-25T20:42:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:42:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ae736451-33b4-4280-b078-499058ffab98 2024-08-25T20:42:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:42:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:42:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:42:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:42:28' 2024-08-25T20:42:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:42:36.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:42:36.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:42:36.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:42:36.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:42:36.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:36.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:42:36.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:36.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:42:36.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:42:36.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:42:36.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:42:36.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:36.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-08-25 20:42:28' 2024-08-25T20:42:36.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:36.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:36.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:42:36.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:42:36.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:42:28' 2024-08-25T20:42:36.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:42:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:42:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:42:37.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:42:28' 2024-08-25T20:42:37.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:42:37.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:42:37.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:42:28' 2024-08-25T20:42:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:42:37.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:42:37.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:42:37.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:42:37.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:42:37.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:37.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:42:37.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary2 2024-08-25T20:42:37.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:42:37.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:42:37.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:42:37.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:42:37.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-08-25T20:42:37.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:42:37.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:42:37.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:42:37.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:42:37.565 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:42:37.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:42:37.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:42:37.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-08-25T20:42:37.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:42:37.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:42:37.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:42:37.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:42:37.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-08-25T20:42:37.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-08-25T20:42:37.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:42:37.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:42:37.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-08-25T20:42:37.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:42:37.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:42:37.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-08-25T20:42:37.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd5 2024-08-25T20:42:37.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:42:37.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:42:37.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd6 2024-08-25T20:42:37.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:42:37.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:42:38.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd8 2024-08-25T20:42:38.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:42:38.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:42:38.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd7 2024-08-25T20:42:38.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:42:38.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:42:38.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd9 2024-08-25T20:42:38.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:42:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:43:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=d79c8dfd929175a28d7fff38b895d028 2024-08-25T20:43:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-08-25T20:43:21.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=7fbbfe32b9605446f48a239b1b8c5c05 2024-08-25T20:43:21.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-08-25T20:43:21.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=db7cde1e00ab7a765d88eafee2e7d8aa 2024-08-25T20:43:21.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-08-25T20:43:21.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ d79c8dfd929175a28d7fff38b895d028 != \d\7\9\c\8\d\f\d\9\2\9\1\7\5\a\2\8\d\7\f\f\f\3\8\b\8\9\5\d\0\2\8 ]] 2024-08-25T20:43:22.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 7fbbfe32b9605446f48a239b1b8c5c05 != \7\f\b\b\f\e\3\2\b\9\6\0\5\4\4\6\f\4\8\a\2\3\9\b\1\b\8\c\5\c\0\5 ]] 2024-08-25T20:43:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ db7cde1e00ab7a765d88eafee2e7d8aa != \d\b\7\c\d\e\1\e\0\0\a\b\7\a\7\6\5\d\8\8\e\a\f\e\e\2\e\7\d\8\a\a ]] 2024-08-25T20:43:24.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=1f01bbd50ae44e4591fbb57c893e4c47 2024-08-25T20:43:24.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-08-25T20:43:24.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 1f01bbd50ae44e4591fbb57c893e4c47 != \1\f\0\1\b\b\d\5\0\a\e\4\4\e\4\5\9\1\f\b\b\5\7\c\8\9\3\e\4\c\4\7 ]] 2024-08-25T20:43:27.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=52471b9404c85f659911705ff902143c 2024-08-25T20:43:27.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-08-25T20:43:28.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 52471b9404c85f659911705ff902143c != \5\2\4\7\1\b\9\4\0\4\c\8\5\f\6\5\9\9\1\1\7\0\5\f\f\9\0\2\1\4\3\c ]] 2024-08-25T20:43:29.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=b996c713be26eb2e3ae5400e70062551 2024-08-25T20:43:29.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-08-25T20:43:29.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=0f35411bf327e153ff073fc7456a30c5 2024-08-25T20:43:29.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-08-25T20:43:29.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ b996c713be26eb2e3ae5400e70062551 != \b\9\9\6\c\7\1\3\b\e\2\6\e\b\2\e\3\a\e\5\4\0\0\e\7\0\0\6\2\5\5\1 ]] 2024-08-25T20:43:29.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 0f35411bf327e153ff073fc7456a30c5 != \0\f\3\5\4\1\1\b\f\3\2\7\e\1\5\3\f\f\0\7\3\f\c\7\4\5\6\a\3\0\c\5 ]] 2024-08-25T20:43:29.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=effc958b150ce8355bbb2c0d3ef539b7 2024-08-25T20:43:29.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-08-25T20:43:29.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ effc958b150ce8355bbb2c0d3ef539b7 != \e\f\f\c\9\5\8\b\1\5\0\c\e\8\3\5\5\b\b\b\2\c\0\d\3\e\f\5\3\9\b\7 ]] 2024-08-25T20:43:30.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=a48c78bcf94b7f4150eb518e716232c9 2024-08-25T20:43:30.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-08-25T20:43:30.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ a48c78bcf94b7f4150eb518e716232c9 != \a\4\8\c\7\8\b\c\f\9\4\b\7\f\4\1\5\0\e\b\5\1\8\e\7\1\6\2\3\2\c\9 ]] 2024-08-25T20:43:30.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=e52a8be352b807af7d8c0938dbb38453 2024-08-25T20:43:30.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-08-25T20:43:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ e52a8be352b807af7d8c0938dbb38453 != \e\5\2\a\8\b\e\3\5\2\b\8\0\7\a\f\7\d\8\c\0\9\3\8\d\b\b\3\8\4\5\3 ]] 2024-08-25T20:43:30.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:43:30.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 102167 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 102169 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 102171 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 102173 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 102175 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 102177 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 102179 2024-08-25T20:43:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 102181 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 102182 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-08-25T20:43:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:43:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:43:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-08-25T20:43:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:43:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:43:31.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:43:31.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8d09bd3d-e631-4259-85a7-7df19d5d05a5 2024-08-25T20:43:31.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:43:31.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3418112.0,"bytes_per_snapshot":68362240.0,"last_snapshot_bytes":68362240,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"} 2024-08-25T20:43:31.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:43:31.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:43:28' 2024-08-25T20:43:31.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:43:31.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:43:31.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:43:31.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3418112.0,"bytes_per_snapshot":68362240.0,"last_snapshot_bytes":68362240,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"} 2024-08-25T20:43:31.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:43:31.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:31.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:43:31.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:31.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:43:31.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:43:31.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:43:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:43:31.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:31.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-08-25 20:43:28' 2024-08-25T20:43:31.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:31.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:31.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:43:31.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:31.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:43:28' 2024-08-25T20:43:31.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:31.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:31.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:43:31.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary18d09bd3d-e631-4259-85a7-7df19d5d05a5up+replayingreplaying, {"bytes_per_second":3418112.0,"bytes_per_snapshot":68362240.0,"last_snapshot_bytes":68362240,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+replayingreplaying, {"bytes_per_second":3179929.6,"bytes_per_snapshot":63598592.0,"last_snapshot_bytes":63598592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+replayingreplaying, {"bytes_per_second":3240755.2,"bytes_per_snapshot":64815104.0,"last_snapshot_bytes":64815104,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+replayingreplaying, {"bytes_per_second":3246899.2,"bytes_per_snapshot":64937984.0,"last_snapshot_bytes":64937984,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618555,"remote_snapshot_timestamp":1724618555,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+replayingreplaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+replayingreplaying, {"bytes_per_second":3234611.2,"bytes_per_snapshot":64692224.0,"last_snapshot_bytes":64692224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+replayingreplaying, {"bytes_per_second":3389849.6,"bytes_per_snapshot":67796992.0,"last_snapshot_bytes":67796992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+replayingreplaying, {"bytes_per_second":3220889.6,"bytes_per_snapshot":64417792.0,"last_snapshot_bytes":64417792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618552,"remote_snapshot_timestamp":1724618552,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+replayingreplaying, {"bytes_per_second":3411558.4,"bytes_per_snapshot":68231168.0,"last_snapshot_bytes":68231168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+replayingreplaying, {"bytes_per_second":3304857.6,"bytes_per_snapshot":66097152.0,"last_snapshot_bytes":66097152,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28' 2024-08-25T20:43:31.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:43:31.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:43:31.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:43:31.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:43:31.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:31.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:43:31.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:31.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary1 2024-08-25T20:43:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:43:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:43:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-08-25T20:43:31.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-08-25T20:43:32.122 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:32.120+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4c3bd40 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:43:32.122 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:32.120+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972500 [3/8d09bd3d-e631-4259-85a7-7df19d5d05a5] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:43:33.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:43:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-08-25T20:43:33.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:33.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:33.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:43:33.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:33.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:43:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary1 2024-08-25T20:43:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary1 2024-08-25T20:43:33.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:43:33.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:43:33.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-08-25T20:43:33.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:33.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:33.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:43:33.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:43:33.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-08-25T20:43:33.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:33.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:33.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:43:33.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:33.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:33.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:33.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:33.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:43:34.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-08-25T20:43:34.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:34.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:34.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:43:34.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:34.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:34.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:34.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:34.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:34.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:43:34.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:43:34.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ae736451-33b4-4280-b078-499058ffab98 2024-08-25T20:43:34.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:43:34.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3240755.2,"bytes_per_snapshot":64815104.0,"last_snapshot_bytes":64815104,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"} 2024-08-25T20:43:34.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:43:34.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:43:28' 2024-08-25T20:43:34.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:43:34.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:43:34.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:43:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3240755.2,"bytes_per_snapshot":64815104.0,"last_snapshot_bytes":64815104,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"} 2024-08-25T20:43:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:43:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:43:34.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:34.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:43:34.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:43:34.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:43:34.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:43:34.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:34.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-08-25 20:43:28' 2024-08-25T20:43:34.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:34.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:34.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:43:34.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:34.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:43:28' 2024-08-25T20:43:34.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:34.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:34.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:43:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING941904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+replayingreplaying, {"bytes_per_second":3179929.6,"bytes_per_snapshot":63598592.0,"last_snapshot_bytes":63598592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary2ae736451-33b4-4280-b078-499058ffab98up+replayingreplaying, {"bytes_per_second":3240755.2,"bytes_per_snapshot":64815104.0,"last_snapshot_bytes":64815104,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+replayingreplaying, {"bytes_per_second":3246899.2,"bytes_per_snapshot":64937984.0,"last_snapshot_bytes":64937984,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618555,"remote_snapshot_timestamp":1724618555,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+replayingreplaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+replayingreplaying, {"bytes_per_second":3234611.2,"bytes_per_snapshot":64692224.0,"last_snapshot_bytes":64692224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+replayingreplaying, {"bytes_per_second":3389849.6,"bytes_per_snapshot":67796992.0,"last_snapshot_bytes":67796992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+replayingreplaying, {"bytes_per_second":3220889.6,"bytes_per_snapshot":64417792.0,"last_snapshot_bytes":64417792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618552,"remote_snapshot_timestamp":1724618552,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+replayingreplaying, {"bytes_per_second":3411558.4,"bytes_per_snapshot":68231168.0,"last_snapshot_bytes":68231168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+replayingreplaying, {"bytes_per_second":3304857.6,"bytes_per_snapshot":66097152.0,"last_snapshot_bytes":66097152,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28' 2024-08-25T20:43:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:43:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:43:35.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:35.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:43:35.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:43:35.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:35.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:43:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary2 2024-08-25T20:43:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:43:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:43:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-08-25T20:43:35.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-08-25T20:43:35.253 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:35.252+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4e68d00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:43:35.253 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:35.253+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972000 [3/ae736451-33b4-4280-b078-499058ffab98] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:43:36.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-08-25T20:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:43:36.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:36.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:43:36.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary2 2024-08-25T20:43:36.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:36.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary2 2024-08-25T20:43:36.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:43:36.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:43:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-08-25T20:43:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:43:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:43:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-08-25T20:43:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:43:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:43:37.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-08-25T20:43:37.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:37.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:37.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:43:37.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:37.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:37.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:37.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:37.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:37.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:43:37.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:43:37.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 629954bc-57dc-455c-84fe-8764b2a92c2e 2024-08-25T20:43:37.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:43:37.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3246899.2,"bytes_per_snapshot":64937984.0,"last_snapshot_bytes":64937984,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618555,"remote_snapshot_timestamp":1724618555,"replay_state":"idle"} 2024-08-25T20:43:37.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:43:37.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:43:28' 2024-08-25T20:43:37.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:43:37.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:43:37.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:43:37.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3246899.2,"bytes_per_snapshot":64937984.0,"last_snapshot_bytes":64937984,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618555,"remote_snapshot_timestamp":1724618555,"replay_state":"idle"} 2024-08-25T20:43:37.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:43:37.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:37.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:43:37.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:37.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:43:37.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:43:37.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:43:37.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:43:37.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:37.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-08-25 20:43:28' 2024-08-25T20:43:37.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:37.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:37.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:43:37.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:37.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:43:28' 2024-08-25T20:43:37.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:37.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:37.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:43:38.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING841904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+replayingreplaying, {"bytes_per_second":3179929.6,"bytes_per_snapshot":63598592.0,"last_snapshot_bytes":63598592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary3629954bc-57dc-455c-84fe-8764b2a92c2eup+replayingreplaying, {"bytes_per_second":3246899.2,"bytes_per_snapshot":64937984.0,"last_snapshot_bytes":64937984,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618555,"remote_snapshot_timestamp":1724618555,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+replayingreplaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+replayingreplaying, {"bytes_per_second":3234611.2,"bytes_per_snapshot":64692224.0,"last_snapshot_bytes":64692224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+replayingreplaying, {"bytes_per_second":3389849.6,"bytes_per_snapshot":67796992.0,"last_snapshot_bytes":67796992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+replayingreplaying, {"bytes_per_second":3220889.6,"bytes_per_snapshot":64417792.0,"last_snapshot_bytes":64417792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618552,"remote_snapshot_timestamp":1724618552,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+replayingreplaying, {"bytes_per_second":3411558.4,"bytes_per_snapshot":68231168.0,"last_snapshot_bytes":68231168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+replayingreplaying, {"bytes_per_second":3304857.6,"bytes_per_snapshot":66097152.0,"last_snapshot_bytes":66097152,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28' 2024-08-25T20:43:38.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:43:38.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:43:38.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:38.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:43:38.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:43:38.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:38.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:43:38.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:38.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:38.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:38.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary3 2024-08-25T20:43:38.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:43:38.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:38.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:43:38.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-08-25T20:43:38.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-08-25T20:43:38.248 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:38.247+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8bd380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:43:38.248 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:38.247+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973180 [3/629954bc-57dc-455c-84fe-8764b2a92c2e] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:43:39.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:43:39.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-08-25T20:43:39.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:39.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:39.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:43:39.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:39.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:43:39.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary3 2024-08-25T20:43:39.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:39.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary3 2024-08-25T20:43:39.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:43:39.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:43:39.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-08-25T20:43:39.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:39.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:39.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:43:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:43:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-08-25T20:43:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:43:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:43:40.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-08-25T20:43:40.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:40.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:40.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:43:40.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:40.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:40.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:40.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:40.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:40.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:43:40.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:43:40.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: d8ea85d6-0a14-489d-88c9-253afe4f3a5c 2024-08-25T20:43:40.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:43:40.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"} 2024-08-25T20:43:40.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:43:40.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:43:28' 2024-08-25T20:43:40.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:43:40.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:43:40.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:43:40.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"} 2024-08-25T20:43:40.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:43:40.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:40.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:43:40.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:40.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:43:40.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:43:40.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:43:40.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:43:40.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:40.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-08-25 20:43:28' 2024-08-25T20:43:40.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:40.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:40.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:43:40.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:40.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:43:28' 2024-08-25T20:43:40.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:40.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:40.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:43:41.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING741904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+replayingreplaying, {"bytes_per_second":3179929.6,"bytes_per_snapshot":63598592.0,"last_snapshot_bytes":63598592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary4d8ea85d6-0a14-489d-88c9-253afe4f3a5cup+replayingreplaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+replayingreplaying, {"bytes_per_second":3234611.2,"bytes_per_snapshot":64692224.0,"last_snapshot_bytes":64692224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+replayingreplaying, {"bytes_per_second":3389849.6,"bytes_per_snapshot":67796992.0,"last_snapshot_bytes":67796992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+replayingreplaying, {"bytes_per_second":3220889.6,"bytes_per_snapshot":64417792.0,"last_snapshot_bytes":64417792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618552,"remote_snapshot_timestamp":1724618552,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+replayingreplaying, {"bytes_per_second":3411558.4,"bytes_per_snapshot":68231168.0,"last_snapshot_bytes":68231168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+replayingreplaying, {"bytes_per_second":3304857.6,"bytes_per_snapshot":66097152.0,"last_snapshot_bytes":66097152,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28' 2024-08-25T20:43:41.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:43:41.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:43:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:43:41.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:43:41.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:41.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:43:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary4 2024-08-25T20:43:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:43:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:43:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-08-25T20:43:41.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-08-25T20:43:41.381 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:41.380+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4c3b860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:43:41.381 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:41.380+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973900 [3/d8ea85d6-0a14-489d-88c9-253afe4f3a5c] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:43:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:43:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-08-25T20:43:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:43:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:43:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary4 2024-08-25T20:43:42.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:42.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary4 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:42.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:42.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:42.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:43:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-08-25T20:43:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:43:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:43.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:43.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:43.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:43:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:43:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8ea7a427-cfbf-4f64-b72c-363c5306e09f 2024-08-25T20:43:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:43:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3234611.2,"bytes_per_snapshot":64692224.0,"last_snapshot_bytes":64692224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"} 2024-08-25T20:43:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:43:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:43:28' 2024-08-25T20:43:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:43:43.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:43:43.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:43:43.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3234611.2,"bytes_per_snapshot":64692224.0,"last_snapshot_bytes":64692224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"} 2024-08-25T20:43:43.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:43:43.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:43.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:43:43.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:43.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:43:43.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:43:43.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:43:43.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:43:43.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:43.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-08-25 20:43:28' 2024-08-25T20:43:43.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:43.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:43.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:43:43.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:43.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:43:28' 2024-08-25T20:43:43.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:43.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:43.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:43:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING641904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+replayingreplaying, {"bytes_per_second":3179929.6,"bytes_per_snapshot":63598592.0,"last_snapshot_bytes":63598592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary58ea7a427-cfbf-4f64-b72c-363c5306e09fup+replayingreplaying, {"bytes_per_second":3234611.2,"bytes_per_snapshot":64692224.0,"last_snapshot_bytes":64692224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+replayingreplaying, {"bytes_per_second":3389849.6,"bytes_per_snapshot":67796992.0,"last_snapshot_bytes":67796992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+replayingreplaying, {"bytes_per_second":3220889.6,"bytes_per_snapshot":64417792.0,"last_snapshot_bytes":64417792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618552,"remote_snapshot_timestamp":1724618552,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+replayingreplaying, {"bytes_per_second":3411558.4,"bytes_per_snapshot":68231168.0,"last_snapshot_bytes":68231168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+replayingreplaying, {"bytes_per_second":3304857.6,"bytes_per_snapshot":66097152.0,"last_snapshot_bytes":66097152,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28' 2024-08-25T20:43:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:43:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:43:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:44.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:43:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:43:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:44.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:43:44.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:44.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:44.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary5 2024-08-25T20:43:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:43:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:43:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-08-25T20:43:44.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-08-25T20:43:44.269 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:44.268+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3663d40 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:43:44.269 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:44.268+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972280 [3/8ea7a427-cfbf-4f64-b72c-363c5306e09f] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:43:45.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:43:45.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-08-25T20:43:45.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:45.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:45.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:43:45.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:45.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:43:45.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary5 2024-08-25T20:43:45.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:45.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary5 2024-08-25T20:43:45.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:43:45.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:43:45.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-08-25T20:43:45.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:45.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:45.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:43:45.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:43:45.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-08-25T20:43:45.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:45.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:45.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:43:45.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:45.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:45.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:45.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:45.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:43:46.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-08-25T20:43:46.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:46.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:46.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:43:46.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:46.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:46.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:46.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:46.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:46.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:43:46.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:43:46.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7a92202a-68ae-4d01-928a-945468468e5c 2024-08-25T20:43:46.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:43:46.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3389849.6,"bytes_per_snapshot":67796992.0,"last_snapshot_bytes":67796992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"} 2024-08-25T20:43:46.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:43:46.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:43:28' 2024-08-25T20:43:46.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:43:46.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:43:46.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:43:46.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3389849.6,"bytes_per_snapshot":67796992.0,"last_snapshot_bytes":67796992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"} 2024-08-25T20:43:46.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:43:46.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:46.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:43:46.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:46.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:43:46.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:43:46.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:43:46.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:43:46.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-08-25 20:43:28' 2024-08-25T20:43:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:43:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:43:28' 2024-08-25T20:43:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:43:46.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING541904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+replayingreplaying, {"bytes_per_second":3179929.6,"bytes_per_snapshot":63598592.0,"last_snapshot_bytes":63598592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary67a92202a-68ae-4d01-928a-945468468e5cup+replayingreplaying, {"bytes_per_second":3389849.6,"bytes_per_snapshot":67796992.0,"last_snapshot_bytes":67796992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+replayingreplaying, {"bytes_per_second":3220889.6,"bytes_per_snapshot":64417792.0,"last_snapshot_bytes":64417792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618552,"remote_snapshot_timestamp":1724618552,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+replayingreplaying, {"bytes_per_second":3411558.4,"bytes_per_snapshot":68231168.0,"last_snapshot_bytes":68231168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+replayingreplaying, {"bytes_per_second":3304857.6,"bytes_per_snapshot":66097152.0,"last_snapshot_bytes":66097152,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28' 2024-08-25T20:43:46.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:43:46.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:43:46.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:46.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:43:46.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:43:46.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:46.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:43:46.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary6 2024-08-25T20:43:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:43:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:43:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-08-25T20:43:46.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-08-25T20:43:47.070 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:47.069+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4c3a000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:43:47.070 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:47.069+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972f00 [3/7a92202a-68ae-4d01-928a-945468468e5c] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:43:48.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:43:48.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-08-25T20:43:48.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:48.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:48.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:43:48.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:48.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:43:48.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary6 2024-08-25T20:43:48.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:48.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary6 2024-08-25T20:43:48.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:43:48.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:43:48.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-08-25T20:43:48.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:48.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:48.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:43:48.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:43:48.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-08-25T20:43:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:43:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:43:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-08-25T20:43:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:43:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:49.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:49.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:49.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:49.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:49.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:43:49.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:43:49.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7da99107-e1ee-4201-a999-13a8eec52934 2024-08-25T20:43:49.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:43:49.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3220889.6,"bytes_per_snapshot":64417792.0,"last_snapshot_bytes":64417792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618552,"remote_snapshot_timestamp":1724618552,"replay_state":"idle"} 2024-08-25T20:43:49.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:43:49.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:43:28' 2024-08-25T20:43:49.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:43:49.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:43:49.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:43:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3220889.6,"bytes_per_snapshot":64417792.0,"last_snapshot_bytes":64417792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618552,"remote_snapshot_timestamp":1724618552,"replay_state":"idle"} 2024-08-25T20:43:49.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:43:49.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:49.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:43:49.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:49.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:43:49.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:43:49.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:43:49.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:43:49.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:49.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-08-25 20:43:28' 2024-08-25T20:43:49.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:49.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:49.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:43:49.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:49.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:43:28' 2024-08-25T20:43:49.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:49.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:49.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:43:50.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING441904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+replayingreplaying, {"bytes_per_second":3179929.6,"bytes_per_snapshot":63598592.0,"last_snapshot_bytes":63598592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary77da99107-e1ee-4201-a999-13a8eec52934up+replayingreplaying, {"bytes_per_second":3220889.6,"bytes_per_snapshot":64417792.0,"last_snapshot_bytes":64417792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618552,"remote_snapshot_timestamp":1724618552,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+replayingreplaying, {"bytes_per_second":3411558.4,"bytes_per_snapshot":68231168.0,"last_snapshot_bytes":68231168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+replayingreplaying, {"bytes_per_second":3304857.6,"bytes_per_snapshot":66097152.0,"last_snapshot_bytes":66097152,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28' 2024-08-25T20:43:50.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:43:50.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:43:50.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:50.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:43:50.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:43:50.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:50.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:43:50.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:50.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:50.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:50.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary7 2024-08-25T20:43:50.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:43:50.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:50.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:43:50.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-08-25T20:43:50.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-08-25T20:43:50.251 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:50.250+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade7c6000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:43:50.251 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:50.250+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973680 [3/7da99107-e1ee-4201-a999-13a8eec52934] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:43:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:43:51.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-08-25T20:43:51.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:51.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:51.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:43:51.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:51.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:43:51.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary7 2024-08-25T20:43:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary7 2024-08-25T20:43:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:43:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:43:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:43:52.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-08-25T20:43:52.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:52.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:52.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:43:52.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:52.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:52.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:52.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:52.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:52.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:43:52.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:43:52.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 72b47598-0266-4d1c-81e0-f751f633b29c 2024-08-25T20:43:52.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:43:52.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3411558.4,"bytes_per_snapshot":68231168.0,"last_snapshot_bytes":68231168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"} 2024-08-25T20:43:52.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:43:52.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:43:28' 2024-08-25T20:43:52.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:43:52.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:43:52.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:43:52.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3411558.4,"bytes_per_snapshot":68231168.0,"last_snapshot_bytes":68231168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"} 2024-08-25T20:43:52.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:43:52.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:52.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:43:52.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:52.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:43:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:43:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:43:52.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:43:52.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:52.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-08-25 20:43:28' 2024-08-25T20:43:52.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:52.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:52.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:43:52.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:52.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:43:28' 2024-08-25T20:43:52.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:52.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:52.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:43:52.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING341904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+replayingreplaying, {"bytes_per_second":3179929.6,"bytes_per_snapshot":63598592.0,"last_snapshot_bytes":63598592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary872b47598-0266-4d1c-81e0-f751f633b29cup+replayingreplaying, {"bytes_per_second":3411558.4,"bytes_per_snapshot":68231168.0,"last_snapshot_bytes":68231168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+replayingreplaying, {"bytes_per_second":3304857.6,"bytes_per_snapshot":66097152.0,"last_snapshot_bytes":66097152,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28' 2024-08-25T20:43:52.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:43:52.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:43:52.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:52.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:43:52.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:43:52.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:52.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:43:52.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:52.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:52.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:52.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary8 2024-08-25T20:43:52.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:43:52.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:52.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:43:52.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-08-25T20:43:52.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-08-25T20:43:53.066 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:53.065+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adf4c9ba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:43:53.066 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:53.065+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ae365e000 [3/72b47598-0266-4d1c-81e0-f751f633b29c] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:43:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:43:54.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-08-25T20:43:54.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:54.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:54.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:43:54.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:54.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:43:54.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary8 2024-08-25T20:43:54.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:54.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary8 2024-08-25T20:43:54.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:43:54.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:43:54.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:54.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:43:55.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-08-25T20:43:55.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:55.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:55.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:43:55.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:55.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:55.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:55.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:55.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:55.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:43:55.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:43:55.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3d225f7-8cf1-4273-85ce-9973dfd95e9a 2024-08-25T20:43:55.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:43:55.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3304857.6,"bytes_per_snapshot":66097152.0,"last_snapshot_bytes":66097152,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"} 2024-08-25T20:43:55.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:43:55.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:43:28' 2024-08-25T20:43:55.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:43:55.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:43:55.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:43:55.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3304857.6,"bytes_per_snapshot":66097152.0,"last_snapshot_bytes":66097152,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"} 2024-08-25T20:43:55.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:43:55.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:55.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:43:55.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:55.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:43:55.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:43:55.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:43:55.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:43:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-08-25 20:43:28' 2024-08-25T20:43:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:43:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:43:28' 2024-08-25T20:43:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:43:55.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING241904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+replayingreplaying, {"bytes_per_second":3179929.6,"bytes_per_snapshot":63598592.0,"last_snapshot_bytes":63598592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28image-primary9c3d225f7-8cf1-4273-85ce-9973dfd95e9aup+replayingreplaying, {"bytes_per_second":3304857.6,"bytes_per_snapshot":66097152.0,"last_snapshot_bytes":66097152,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618557,"remote_snapshot_timestamp":1724618557,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:28' 2024-08-25T20:43:55.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:43:55.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:43:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:28' 2024-08-25T20:43:55.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:43:55.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:43:55.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:55.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:43:55.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:55.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:55.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:55.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary9 2024-08-25T20:43:55.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:43:55.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:55.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:43:55.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-08-25T20:43:55.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-08-25T20:43:56.077 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:56.076+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade975860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:43:56.077 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:56.076+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972780 [3/c3d225f7-8cf1-4273-85ce-9973dfd95e9a] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:43:57.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:43:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-08-25T20:43:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:43:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:43:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary9 2024-08-25T20:43:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary9 2024-08-25T20:43:57.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:43:57.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:43:57.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:43:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:43:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-08-25T20:43:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:43:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:43:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:43:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:43:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:43:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1d9ccc9-fc58-49ab-9a15-08aef159ef5f 2024-08-25T20:43:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:43:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63598592.0,"last_snapshot_bytes":63598592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"} 2024-08-25T20:43:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:43:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:43:58' 2024-08-25T20:43:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:43:58.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:43:58.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:43:58.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63598592.0,"last_snapshot_bytes":63598592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"} 2024-08-25T20:43:58.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:43:58.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:58.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:43:58.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:58.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:43:58.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:43:58.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:43:58.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:43:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:58' 2024-08-25T20:43:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-08-25 20:43:58' 2024-08-25T20:43:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:43:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:43:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:43:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:43:58' 2024-08-25T20:43:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:43:58.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:43:58.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:43:58.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING141904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c1d9ccc9-fc58-49ab-9a15-08aef159ef5fup+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63598592.0,"last_snapshot_bytes":63598592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724618551,"remote_snapshot_timestamp":1724618551,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:43:58' 2024-08-25T20:43:58.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:43:58.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:43:58.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:43:58' 2024-08-25T20:43:58.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:43:58.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:43:58.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:43:58.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:43:58.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:43:58.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:58.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:43:58.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary10 2024-08-25T20:43:58.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:43:58.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:43:58.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:43:58.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-08-25T20:43:58.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-08-25T20:43:59.078 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:59.077+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae36ffa00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:43:59.079 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:43:59.077+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ae365f900 [3/c1d9ccc9-fc58-49ab-9a15-08aef159ef5f] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:44:00.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:44:00.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-08-25T20:44:00.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:00.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:00.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:00.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:44:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:44:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary10 2024-08-25T20:44:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:44:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary10 2024-08-25T20:44:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..10} 2024-08-25T20:44:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS=() 2024-08-25T20:44:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS=() 2024-08-25T20:44:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS=() 2024-08-25T20:44:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS=() 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RET=0 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary1 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:44:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-08-25T20:44:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:44:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:44:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:44:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-08-25T20:44:00.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:44:00.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:00.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:44:00.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:44:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-08-25T20:44:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-08-25T20:44:01.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:44:01.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-08-25T20:44:01.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary1': 2024-08-25T20:44:01.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:44:01.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:44:01.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:44:01.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 33b8d06d2ccc 2024-08-25T20:44:01.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.33b8d06d2ccc 2024-08-25T20:44:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:44:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:44:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:44:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:44:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:44:00 2024 2024-08-25T20:44:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:44:00 2024 2024-08-25T20:44:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:44:00 2024 2024-08-25T20:44:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:44:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:44:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 0cd02d4a-436f-4aa7-aa9d-42409daf184b 2024-08-25T20:44:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:44:01.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:44:01.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:44:01.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-08-25T20:44:01.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd0 2024-08-25T20:44:01.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:44:01.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-08-25T20:44:01.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:44:01.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:44:01.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:44:01.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 836e5179-1db2-4dc9-aca9-f0e41f5aaca8 2024-08-25T20:44:01.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:44:01.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:44:01.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:01.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:44:01.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:44:02.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:44:02.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:44:02.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:02.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary1 2024-08-25T20:44:02.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-08-25T20:44:02.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:44:02.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary1 2024-08-25T20:44:02.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:44:02.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-08-25T20:44:02.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:02.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:02.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:44:02.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary2 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary1 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary1 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:44:02.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-08-25T20:44:02.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:44:02.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:44:02.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-08-25T20:44:02.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:44:02.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:02.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:44:02.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:44:02.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-08-25T20:44:02.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:02.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:02.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:02.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:02.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-08-25T20:44:02.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:44:02.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-08-25T20:44:02.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:44:02.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2314 2024-08-25T20:44:02.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-08-25T20:44:02.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary2': 2024-08-25T20:44:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:44:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:44:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:44:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 33c8d46b87d7 2024-08-25T20:44:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.33c8d46b87d7 2024-08-25T20:44:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:44:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:44:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:44:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:44:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:44:02 2024 2024-08-25T20:44:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:44:02 2024 2024-08-25T20:44:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:44:02 2024 2024-08-25T20:44:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:44:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:44:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 463c84e4-50f5-4605-b0bd-7e42905093fc 2024-08-25T20:44:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:44:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:44:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:44:02.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-08-25T20:44:02.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd1 2024-08-25T20:44:02.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:44:02.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-08-25T20:44:02.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:44:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:44:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:44:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: f85fb809-cedc-4f8c-a266-4d234a5495f6 2024-08-25T20:44:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:44:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:44:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:44:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:44:02.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:44:02.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:44:02.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:02.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary2 2024-08-25T20:44:02.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-08-25T20:44:02.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:44:02.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary2 2024-08-25T20:44:02.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:44:02.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-08-25T20:44:02.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:02.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:02.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:02.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:02.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:02.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:02.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary2 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary3 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary2 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:44:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-08-25T20:44:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:02.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:02.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:44:02.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:44:02.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:44:02.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:02.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-08-25T20:44:03.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:44:03.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:03.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:44:03.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:44:03.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-08-25T20:44:03.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:03.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:03.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:03.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:03.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-08-25T20:44:03.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:44:03.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-08-25T20:44:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:44:03.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2316 2024-08-25T20:44:03.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-08-25T20:44:03.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary3': 2024-08-25T20:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 33d7845488ea 2024-08-25T20:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.33d7845488ea 2024-08-25T20:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:44:03 2024 2024-08-25T20:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:44:03 2024 2024-08-25T20:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:44:03 2024 2024-08-25T20:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 3b1376ab-05fd-460d-9be4-66b67ff501ea 2024-08-25T20:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:44:03.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:44:03.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:44:03.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-08-25T20:44:03.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd2 2024-08-25T20:44:03.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:44:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-08-25T20:44:03.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:44:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:44:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:44:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 22808b38-a868-4df8-bb37-f2a774ef4e0d 2024-08-25T20:44:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:44:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:44:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:44:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:44:03.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:44:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:44:03.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:03.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary3 2024-08-25T20:44:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-08-25T20:44:03.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:44:04.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary3 2024-08-25T20:44:04.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:44:04.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-08-25T20:44:04.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:04.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:04.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:04.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:04.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:04.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:04.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:04.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:04.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:04.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:44:04.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:44:04.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary3 2024-08-25T20:44:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:44:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary4 2024-08-25T20:44:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-08-25T20:44:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary3 2024-08-25T20:44:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-08-25T20:44:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:04.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-08-25T20:44:04.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:44:04.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:04.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:44:04.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:44:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-08-25T20:44:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-08-25T20:44:04.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:44:04.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-08-25T20:44:04.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:44:04.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2318 2024-08-25T20:44:04.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:04.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-08-25T20:44:04.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary4': 2024-08-25T20:44:04.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:44:04.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:44:04.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:44:04.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 33e6ac54fadf 2024-08-25T20:44:04.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.33e6ac54fadf 2024-08-25T20:44:04.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:44:04.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:44:04.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:44:04.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:44:04.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:44:04 2024 2024-08-25T20:44:04.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:44:04 2024 2024-08-25T20:44:04.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:44:04 2024 2024-08-25T20:44:04.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:44:04.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:44:04.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: c1bee555-24e4-4fa8-9d86-30eac06d0a76 2024-08-25T20:44:04.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:44:04.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:44:04.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:44:04.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-08-25T20:44:04.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd3 2024-08-25T20:44:04.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:44:04.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-08-25T20:44:04.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:44:04.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:44:04.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:44:04.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 0a1ed74d-efc9-456c-8e1d-c3011987a5a0 2024-08-25T20:44:04.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:44:04.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:44:04.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:04.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:44:04.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:44:04.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:44:04.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:44:04.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:04.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary4 2024-08-25T20:44:04.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-08-25T20:44:04.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:44:05.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary4 2024-08-25T20:44:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:44:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-08-25T20:44:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:44:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary5 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:44:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary4 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-08-25T20:44:05.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary4 2024-08-25T20:44:05.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-08-25T20:44:05.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:44:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:44:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:44:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-08-25T20:44:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-08-25T20:44:05.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:44:05.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-08-25T20:44:05.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:05.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:05.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:05.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:05.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:05.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2320 2024-08-25T20:44:05.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:44:05.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:05.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-08-25T20:44:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary5': 2024-08-25T20:44:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:44:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:44:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:44:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 33f56fbc1eaf 2024-08-25T20:44:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.33f56fbc1eaf 2024-08-25T20:44:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:44:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:44:05.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:44:05.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:44:05.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:44:05 2024 2024-08-25T20:44:05.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:44:05 2024 2024-08-25T20:44:05.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:44:05 2024 2024-08-25T20:44:05.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:44:05.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:44:05.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 8e8e6eb4-4c83-4c1e-842f-1191b4695a8c 2024-08-25T20:44:05.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:44:05.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:44:05.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:44:05.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-08-25T20:44:05.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd4 2024-08-25T20:44:05.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:44:05.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-08-25T20:44:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:44:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:44:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:44:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 9fd1f232-10e3-45e6-b70b-1d6a95f886db 2024-08-25T20:44:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:44:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:44:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:44:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:44:05.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:44:06.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:44:06.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:06.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary5 2024-08-25T20:44:06.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-08-25T20:44:06.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:44:06.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary5 2024-08-25T20:44:06.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:44:06.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:44:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary6 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary5 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:06.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary5 2024-08-25T20:44:06.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:06.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:06.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:06.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:06.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:06.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:44:06.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:44:06.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:44:06.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:06.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-08-25T20:44:06.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-08-25T20:44:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:44:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:44:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:44:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-08-25T20:44:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:06.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:06.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:06.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:06.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-08-25T20:44:06.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:44:06.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-08-25T20:44:06.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:06.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:06.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:06.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:06.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:06.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:06.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:44:06.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2323 2024-08-25T20:44:06.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2322 2024-08-25T20:44:06.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:06.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:06.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-08-25T20:44:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary6': 2024-08-25T20:44:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:44:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:44:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:44:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3405c06d2565 2024-08-25T20:44:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3405c06d2565 2024-08-25T20:44:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:44:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:44:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:44:06.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:44:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:44:06 2024 2024-08-25T20:44:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:44:06 2024 2024-08-25T20:44:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:44:06 2024 2024-08-25T20:44:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:44:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:44:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 02f58d00-dc8f-41ff-89a9-c8a6ca6f2c72 2024-08-25T20:44:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:44:06.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:44:06.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:44:06.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-08-25T20:44:06.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd5 2024-08-25T20:44:06.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:44:06.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-08-25T20:44:06.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:44:07.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/26214401048576/2621440 done 2024-08-25T20:44:07.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:44:07.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 633382db-33f5-477f-8415-1baf96ea3091 2024-08-25T20:44:07.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:44:07.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:44:07.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:07.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:44:07.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:44:07.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:44:07.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:44:07.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary6 2024-08-25T20:44:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-08-25T20:44:07.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:44:07.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary6 2024-08-25T20:44:07.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:44:07.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-08-25T20:44:07.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:07.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary7 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:07.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:07.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:44:07.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary6 2024-08-25T20:44:07.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:44:07.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:44:07.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:07.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-08-25T20:44:07.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary6 2024-08-25T20:44:07.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-08-25T20:44:07.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:44:07.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:44:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:44:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-08-25T20:44:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-08-25T20:44:07.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:44:07.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-08-25T20:44:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:07.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2325 2024-08-25T20:44:07.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:07.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:44:07.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2326 2024-08-25T20:44:07.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-08-25T20:44:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary7': 2024-08-25T20:44:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:44:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:44:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 341686fe6bc9 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.341686fe6bc9 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:44:07 2024 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:44:07 2024 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:44:07 2024 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: e92661f6-6b6c-46fb-92be-0c323b12c6ab 2024-08-25T20:44:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:44:07.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:44:07.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:44:07.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-08-25T20:44:08.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd6 2024-08-25T20:44:08.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:44:08.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-08-25T20:44:08.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:44:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:44:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:44:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 30e7009f-f41c-4225-b180-166a1ef7ca2f 2024-08-25T20:44:08.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:44:08.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:44:08.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:08.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:44:08.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:44:08.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:44:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:44:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:08.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary7 2024-08-25T20:44:08.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-08-25T20:44:08.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:44:08.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary7 2024-08-25T20:44:08.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:44:08.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-08-25T20:44:08.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:08.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary8 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary7 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:44:08.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary7 2024-08-25T20:44:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:44:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:44:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:44:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-08-25T20:44:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-08-25T20:44:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:44:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:44:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:44:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-08-25T20:44:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:08.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-08-25T20:44:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2329 2024-08-25T20:44:08.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:44:08.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2328 2024-08-25T20:44:08.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-08-25T20:44:08.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:08.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:08.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:08.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:08.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:08.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:08.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:08.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:08.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:44:08.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-08-25T20:44:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary8': 2024-08-25T20:44:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:44:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:44:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:44:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3428608b9a7f 2024-08-25T20:44:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3428608b9a7f 2024-08-25T20:44:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:44:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:44:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:44:08.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:44:08.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:44:08 2024 2024-08-25T20:44:08.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:44:08 2024 2024-08-25T20:44:08.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:44:08 2024 2024-08-25T20:44:08.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:44:08.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:44:08.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 0f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416 2024-08-25T20:44:08.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:44:08.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:44:08.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:44:08.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-08-25T20:44:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd7 2024-08-25T20:44:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:44:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-08-25T20:44:08.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:44:08.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:44:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:44:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 9f0e1b77-0b28-46b5-b161-d041fd9fa696 2024-08-25T20:44:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:44:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:44:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:44:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:44:09.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:44:09.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:44:09.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:09.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary8 2024-08-25T20:44:09.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-08-25T20:44:09.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:44:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary8 2024-08-25T20:44:09.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:44:09.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary8 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary9 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary8 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-08-25T20:44:09.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:09.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:09.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:09.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:09.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:09.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:09.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:44:09.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:44:09.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:44:09.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:09.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-08-25T20:44:09.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:44:09.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:09.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:44:09.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:44:09.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-08-25T20:44:09.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:09.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:09.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:09.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:09.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-08-25T20:44:09.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:44:09.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-08-25T20:44:09.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2332 2024-08-25T20:44:09.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2331 2024-08-25T20:44:09.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:44:09.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:09.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:09.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-08-25T20:44:09.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:09.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:44:09.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:09.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:09.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:09.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:44:09.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:09.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:09.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:09.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:09.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:09.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:09.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary9': 2024-08-25T20:44:09.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:44:09.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:44:09.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:44:09.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 343aa67513f9 2024-08-25T20:44:09.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.343aa67513f9 2024-08-25T20:44:09.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:44:09.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:44:09.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:44:09.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:44:09.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:44:09 2024 2024-08-25T20:44:09.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:44:09 2024 2024-08-25T20:44:09.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:44:09 2024 2024-08-25T20:44:09.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:44:09.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:44:09.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 60684371-5dff-4e57-aa1c-760325a2a3a3 2024-08-25T20:44:09.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:44:09.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:44:09.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:44:09.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-08-25T20:44:09.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd8 2024-08-25T20:44:09.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:44:09.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-08-25T20:44:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:44:09.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:44:09.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:44:09.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 0c7b7aea-b33c-474e-b38d-6127bf2ae941 2024-08-25T20:44:09.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:44:09.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:44:09.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:09.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:44:09.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:44:09.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:44:10.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:44:10.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:10.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary9 2024-08-25T20:44:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-08-25T20:44:10.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:44:10.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary9 2024-08-25T20:44:10.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:44:10.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-08-25T20:44:10.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:10.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:10.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:10.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:10.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:44:10.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary9 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary10 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:10.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:44:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:44:10.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:44:10.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary9 2024-08-25T20:44:10.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:44:10.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-08-25T20:44:10.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-08-25T20:44:10.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:44:10.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:10.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:44:10.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:44:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-08-25T20:44:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:44:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-08-25T20:44:10.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:44:10.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-08-25T20:44:10.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2335 2024-08-25T20:44:10.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:44:10.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2336 2024-08-25T20:44:10.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2334 2024-08-25T20:44:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:10.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-08-25T20:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:10.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary10': 2024-08-25T20:44:10.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:44:10.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:44:10.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:44:10.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 344e50851993 2024-08-25T20:44:10.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.344e50851993 2024-08-25T20:44:10.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:44:10.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:44:10.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:44:10.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:44:10.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:44:10 2024 2024-08-25T20:44:10.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:44:10 2024 2024-08-25T20:44:10.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:44:10 2024 2024-08-25T20:44:10.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:44:10.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:44:10.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 95f0a085-22a4-4bbc-9daf-f1488ce5a207 2024-08-25T20:44:10.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:44:10.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:44:10.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:44:10.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-08-25T20:44:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:44:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:44:10.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd9 2024-08-25T20:44:10.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:44:10.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-08-25T20:44:10.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:44:11.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:44:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:44:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: dc9ee4f4-f677-46df-8877-f968af65a85a 2024-08-25T20:44:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:44:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:44:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:44:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:44:11.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:44:11.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:44:11.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:44:11.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary10 2024-08-25T20:44:11.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-08-25T20:44:11.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:44:11.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary10 2024-08-25T20:44:11.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary10 2024-08-25T20:44:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:44:11.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 106823 2024-08-25T20:44:11.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary10 2024-08-25T20:44:11.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-08-25T20:44:11.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:44:11.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-08-25T20:44:11.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2340 2024-08-25T20:44:11.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:11.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:11.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:44:11.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:11.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:11.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:11.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:44:11.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2339 2024-08-25T20:44:11.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2338 2024-08-25T20:44:11.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:11.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:11.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:11.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:11.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:11.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:11.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:11.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:12.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:12.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:44:12.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:12.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:12.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:12.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:44:12.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2342 2024-08-25T20:44:12.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2341 2024-08-25T20:44:12.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:12.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:12.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:12.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:12.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:12.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:12.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:12.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:13.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:13.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:13.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:13.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:13.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:13.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:13.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:13.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:44:13.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:13.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:13.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:13.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:44:13.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:13.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:44:13.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:13.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:13.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:13.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:44:13.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2343 2024-08-25T20:44:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2344 2024-08-25T20:44:13.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:13.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:14.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:14.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:44:14.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:14.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:14.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:14.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:44:14.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:14.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:44:14.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:14.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:14.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:14.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:44:14.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2345 2024-08-25T20:44:14.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:14.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:14.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:14.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:14.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:14.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:14.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2347 2024-08-25T20:44:14.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2346 2024-08-25T20:44:14.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:14.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:14.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:15.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:15.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:15.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:15.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:15.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:15.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:44:15.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:15.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:15.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:15.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:44:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2349 2024-08-25T20:44:15.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2350 2024-08-25T20:44:15.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2348 2024-08-25T20:44:15.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:15.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:15.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2352 2024-08-25T20:44:16.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2351 2024-08-25T20:44:16.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:16.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:16.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:16.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:44:16.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:16.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:16.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:16.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:44:16.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:16.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:17.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2353 2024-08-25T20:44:17.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2354 2024-08-25T20:44:17.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:17.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:44:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:17.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:44:17.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:17.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:44:17.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:17.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:17.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:17.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:44:17.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:17.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:17.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:17.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:17.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:17.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:18.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2355 2024-08-25T20:44:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2356 2024-08-25T20:44:18.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2357 2024-08-25T20:44:18.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:18.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:18.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:18.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:18.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:44:18.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:18.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:18.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:18.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:44:18.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:18.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:44:18.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:18.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:18.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:18.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:44:18.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:18.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:18.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:18.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:18.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:18.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:19.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2359 2024-08-25T20:44:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2358 2024-08-25T20:44:19.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:19.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:44:19.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:19.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:19.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:19.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:44:19.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2360 2024-08-25T20:44:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:19.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:19.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:19.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:20.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:20.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:20.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:20.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:20.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:20.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:20.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2361 2024-08-25T20:44:20.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2362 2024-08-25T20:44:20.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:20.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:44:20.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:20.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:20.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:20.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:44:20.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2363 2024-08-25T20:44:21.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2364 2024-08-25T20:44:21.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:21.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:21.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:21.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:44:21.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:21.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:21.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:21.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:44:21.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:21.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:44:21.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:21.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:21.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:21.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:44:21.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:21.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:21.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:21.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:21.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:21.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:22.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2365 2024-08-25T20:44:22.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2366 2024-08-25T20:44:22.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:22.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:22.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:22.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:44:22.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:22.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:22.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:22.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:44:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:44:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:44:22.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:22.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:22.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:22.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:22.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:22.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:23.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2368 2024-08-25T20:44:23.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:23.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2369 2024-08-25T20:44:23.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:23.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:44:23.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:23.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:23.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:23.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:44:23.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:23.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:23.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:23.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:23.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:23.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:23.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:24.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2367 2024-08-25T20:44:24.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:24.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:24.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:44:24.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:24.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:24.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:24.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:44:24.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:24.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:24.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:24.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:24.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:24.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:24.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2372 2024-08-25T20:44:24.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2370 2024-08-25T20:44:25.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:25.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:25.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:44:25.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:25.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:25.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:25.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:44:25.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:44:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:44:25.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2373 2024-08-25T20:44:25.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:26.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2371 2024-08-25T20:44:26.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:26.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:26.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:44:26.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:26.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:26.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:26.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:44:26.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2375 2024-08-25T20:44:26.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:44:27.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2374 2024-08-25T20:44:27.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:27.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:27.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:27.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:27.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:27.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:27.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:27.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2377 2024-08-25T20:44:27.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:27.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:27.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:44:27.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:27.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:27.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:27.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:44:28.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2376 2024-08-25T20:44:28.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:28.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:28.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:28.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:28.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:28.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:28.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:28.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2379 2024-08-25T20:44:28.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:44:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:28.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:28.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:28.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:44:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2378 2024-08-25T20:44:29.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:29.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:29.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:29.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:29.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:29.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:29.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:29.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2381 2024-08-25T20:44:29.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:29.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:29.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:44:29.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:29.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:29.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:29.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:44:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2380 2024-08-25T20:44:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:30.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:30.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:30.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:44:30.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:30.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:30.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:30.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:44:30.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2383 2024-08-25T20:44:30.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:31.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2382 2024-08-25T20:44:31.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:31.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:31.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:44:31.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:31.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:31.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:31.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:44:31.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:31.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:31.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:31.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:31.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:31.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:31.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2385 2024-08-25T20:44:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2384 2024-08-25T20:44:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:44:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:32.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:44:32.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:32.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:32.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:32.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:32.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:32.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:32.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:32.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2388 2024-08-25T20:44:32.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:33.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2386 2024-08-25T20:44:33.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:33.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:44:33.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:33.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:33.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:33.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:44:33.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2387 2024-08-25T20:44:33.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:33.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2390 2024-08-25T20:44:33.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:34.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:34.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:44:34.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:34.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:34.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:34.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:44:34.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2389 2024-08-25T20:44:34.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:34.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2391 2024-08-25T20:44:34.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:35.779 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:44:35.778+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf1fe400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:44:35.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:35.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:44:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:35.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:44:35.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2393 2024-08-25T20:44:35.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:35.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:35.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:44:35.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:35.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:35.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:35.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:44:36.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2392 2024-08-25T20:44:36.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:36.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2394 2024-08-25T20:44:36.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:36.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:44:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:44:36.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2395 2024-08-25T20:44:36.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:36.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:36.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:36.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:36.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:36.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:36.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:36.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:36.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:44:36.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:36.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:36.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:36.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:44:37.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2398 2024-08-25T20:44:37.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:37.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:37.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:37.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:37.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:37.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:37.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:44:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:44:38.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2396 2024-08-25T20:44:38.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:44:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:44:39.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:39.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:39.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:39.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:39.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:39.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:39.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2399 2024-08-25T20:44:39.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2397 2024-08-25T20:44:39.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:39.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:39.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:39.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:39.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:39.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:39.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:39.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:39.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:44:39.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:39.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:39.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:39.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:44:39.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:39.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2401 2024-08-25T20:44:39.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:40.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2400 2024-08-25T20:44:40.735 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:44:40.734+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae4dbf800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:44:40.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:40.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:40.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:44:40.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:40.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:40.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:40.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:44:40.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2404 2024-08-25T20:44:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:41.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2402 2024-08-25T20:44:41.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2405 2024-08-25T20:44:41.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:41.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2403 2024-08-25T20:44:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:41.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:41.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:44:41.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:41.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:41.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:41.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:44:41.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:42.673 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:44:42.671+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555add606800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:44:42.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2407 2024-08-25T20:44:42.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:42.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:42.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:44:42.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:42.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:42.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:42.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:44:42.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:42.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:42.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:42.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:42.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:42.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:42.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:42.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:44:42.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:42.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:42.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:42.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:44:43.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2406 2024-08-25T20:44:43.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:43.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2409 2024-08-25T20:44:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2408 2024-08-25T20:44:43.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:43.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:43.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:43.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:43.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:43.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:43.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:43.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:43.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:43.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:43.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:43.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:43.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:43.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:44.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:44.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:44:44.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:44.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:44.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:44.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:44:44.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:44.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:44:44.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:44.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:44.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:44.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:44:44.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:44.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2412 2024-08-25T20:44:44.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:44.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2411 2024-08-25T20:44:44.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:45.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2410 2024-08-25T20:44:45.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:45.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:44:45.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:45.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:45.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:45.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:44:45.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:45.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2413 2024-08-25T20:44:45.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:46.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:46.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:44:46.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:46.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:46.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:46.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:44:46.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:46.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:46.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:46.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:46.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:46.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:46.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:46.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:44:46.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:46.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:46.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:46.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:44:47.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2415 2024-08-25T20:44:47.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2414 2024-08-25T20:44:47.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:47.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:47.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2417 2024-08-25T20:44:47.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:47.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:47.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:47.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:47.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:47.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:47.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:47.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:47.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:47.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:47.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:47.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:47.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:48.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2416 2024-08-25T20:44:48.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:48.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:48.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:44:48.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:48.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:48.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:48.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:44:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2420 2024-08-25T20:44:48.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:44:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:44:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2419 2024-08-25T20:44:49.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2418 2024-08-25T20:44:49.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:49.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:49.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2422 2024-08-25T20:44:49.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2423 2024-08-25T20:44:49.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:49.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:50.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2421 2024-08-25T20:44:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:50.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:50.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:50.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:44:50.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:50.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:50.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:50.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:44:50.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:50.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:44:50.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:50.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:50.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:50.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:44:51.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2426 2024-08-25T20:44:51.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:51.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2424 2024-08-25T20:44:51.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2425 2024-08-25T20:44:51.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:51.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:44:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:44:51.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:51.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:51.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:51.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:51.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:51.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:52.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:52.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:52.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:52.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:52.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:52.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:52.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:52.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:44:52.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:52.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:52.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:52.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:44:52.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2428 2024-08-25T20:44:52.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:52.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:44:52.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:52.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:52.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:52.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:44:52.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:52.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:44:52.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:52.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:52.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:52.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:44:52.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2427 2024-08-25T20:44:52.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:53.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:53.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:53.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:53.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:53.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:53.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:53.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2432 2024-08-25T20:44:53.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2430 2024-08-25T20:44:53.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2429 2024-08-25T20:44:53.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:53.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:44:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:44:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:54.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:54.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:44:54.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:54.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:54.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:44:54.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:44:54.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2433 2024-08-25T20:44:54.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:55.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2431 2024-08-25T20:44:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:55.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2434 2024-08-25T20:44:55.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:55.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:44:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:44:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:44:55.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:55.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:44:55.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:55.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:55.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:44:55.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:44:56.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2436 2024-08-25T20:44:56.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:56.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:56.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:44:56.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:56.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:56.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:44:56.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:44:56.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:56.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:44:56.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:56.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:56.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:44:56.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:44:56.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:56.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:44:56.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:56.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:56.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:44:56.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:44:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2437 2024-08-25T20:44:57.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2435 2024-08-25T20:44:57.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:57.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:57.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2440 2024-08-25T20:44:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:57.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:57.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:44:57.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:57.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:57.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:44:57.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:44:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2438 2024-08-25T20:44:58.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:58.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:44:58.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:58.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:58.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:44:58.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:44:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2441 2024-08-25T20:44:58.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:58.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:58.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:44:58.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:58.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:58.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:44:58.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:44:58.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:44:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:44:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:44:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:44:59.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:44:59.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:44:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2439 2024-08-25T20:44:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:59.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2442 2024-08-25T20:44:59.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:44:59.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2443 2024-08-25T20:44:59.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:00.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2444 2024-08-25T20:45:00.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:01.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:45:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:45:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:01.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:01.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:45:01.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:01.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:01.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:45:01.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:45:01.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:45:01.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2446 2024-08-25T20:45:01.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:01.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:45:01.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:01.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:01.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:45:01.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:45:01.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:01.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:01.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:45:01.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:01.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:01.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:45:01.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:45:02.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2445 2024-08-25T20:45:02.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:45:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:45:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:45:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:45:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:45:02.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:45:02.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2449 2024-08-25T20:45:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:45:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:45:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:45:03.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:03.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:45:03.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:03.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:03.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:45:03.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:45:03.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:04.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2453 2024-08-25T20:45:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:04.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2448 2024-08-25T20:45:04.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2450 2024-08-25T20:45:04.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2447 2024-08-25T20:45:04.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:04.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:04.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:04.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:04.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:04.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:04.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:04.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:04.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2452 2024-08-25T20:45:04.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:04.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:04.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2451 2024-08-25T20:45:04.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:05.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:05.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:45:05.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:05.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:05.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:45:05.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:45:06.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2455 2024-08-25T20:45:06.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:06.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2456 2024-08-25T20:45:06.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:06.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:06.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:45:06.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:06.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:06.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:45:06.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:45:07.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:07.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:45:07.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:07.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:07.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:45:07.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:45:07.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:07.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:45:07.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:07.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:07.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:45:07.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:45:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:45:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:45:07.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:45:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:45:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:45:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:45:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:45:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:45:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:07.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:45:07.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:07.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:07.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:45:07.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:45:07.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2454 2024-08-25T20:45:07.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:08.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2457 2024-08-25T20:45:08.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:08.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2458 2024-08-25T20:45:08.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:08.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2461 2024-08-25T20:45:08.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:09.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:09.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:09.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:09.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:09.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:09.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:09.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:09.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:45:09.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:09.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:09.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:45:09.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:45:09.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2460 2024-08-25T20:45:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2463 2024-08-25T20:45:09.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:09.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:10.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:10.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:45:10.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:10.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:10.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:45:10.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:45:11.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2459 2024-08-25T20:45:11.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2462 2024-08-25T20:45:11.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:11.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:11.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2465 2024-08-25T20:45:11.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:11.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2464 2024-08-25T20:45:11.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:11.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:11.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:45:11.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:11.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:11.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:45:11.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:45:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:45:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:45:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:45:11.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:11.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:45:11.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:11.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:11.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:45:11.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:45:12.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:12.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:45:12.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:12.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:12.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:45:12.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:45:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:45:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:45:12.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:45:13.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2466 2024-08-25T20:45:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:14.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:14.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:45:14.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:14.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:14.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:45:14.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:45:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:45:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:14.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:14.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:45:14.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:45:14.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:14.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:45:14.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:14.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:14.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:45:14.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:45:14.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:14.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:14.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:14.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:14.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:14.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2468 2024-08-25T20:45:14.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2469 2024-08-25T20:45:14.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2467 2024-08-25T20:45:14.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:14.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:14.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2470 2024-08-25T20:45:15.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2471 2024-08-25T20:45:15.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:15.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:15.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2473 2024-08-25T20:45:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2472 2024-08-25T20:45:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:15.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:16.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2474 2024-08-25T20:45:16.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2475 2024-08-25T20:45:16.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:16.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:16.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:16.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:45:16.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:16.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:16.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:45:16.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:45:17.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2476 2024-08-25T20:45:17.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:17.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:45:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:45:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:45:17.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:17.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:45:17.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:17.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:17.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:45:17.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:45:17.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:17.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:45:17.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:17.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:17.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:45:17.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:45:18.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:18.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:45:18.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:18.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:18.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:45:18.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:45:18.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2479 2024-08-25T20:45:18.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:18.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:45:18.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:18.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:18.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:45:18.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:45:18.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2477 2024-08-25T20:45:18.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2478 2024-08-25T20:45:18.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:18.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:18.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:18.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:18.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:45:18.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:18.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:18.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:45:18.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:45:18.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:18.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:45:18.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:18.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:18.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:45:18.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:45:19.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:19.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:45:19.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:19.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:19.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:45:19.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:45:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:19.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2481 2024-08-25T20:45:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2480 2024-08-25T20:45:19.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:19.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:20.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:20.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:45:20.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:20.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:20.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:45:20.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:45:21.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2483 2024-08-25T20:45:21.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2482 2024-08-25T20:45:21.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:21.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:21.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:21.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:21.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:45:21.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:45:21.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:21.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:21.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:45:21.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:21.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:45:21.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:21.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:45:21.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:45:21.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:21.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:45:21.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:21.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:21.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:45:21.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:45:22.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2485 2024-08-25T20:45:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2484 2024-08-25T20:45:22.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:22.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:22.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2486 2024-08-25T20:45:22.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:22.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:22.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:45:22.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:22.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:22.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:45:22.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:45:22.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:22.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:45:22.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:22.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:22.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:45:22.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:45:23.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2489 2024-08-25T20:45:23.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:24.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2487 2024-08-25T20:45:24.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2488 2024-08-25T20:45:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:45:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:45:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:45:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:45:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:45:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:45:24.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:24.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:25.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2490 2024-08-25T20:45:25.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2491 2024-08-25T20:45:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:25.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:25.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:25.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:25.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:25.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:45:25.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:25.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:25.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:45:25.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:45:25.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:25.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:25.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2493 2024-08-25T20:45:25.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:25.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:45:25.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:25.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:25.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:45:25.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:45:25.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2492 2024-08-25T20:45:25.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:25.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:26.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2495 2024-08-25T20:45:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2494 2024-08-25T20:45:26.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:26.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:45:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:45:26.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:45:27.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:27.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:45:27.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:27.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:27.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:45:27.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:45:27.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:27.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:45:27.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:27.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:27.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:45:27.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:45:27.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2496 2024-08-25T20:45:27.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:28.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:28.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:45:28.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:28.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:28.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:45:28.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:45:28.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:28.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:45:28.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:28.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:28.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:45:28.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:45:28.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:28.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:45:28.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:28.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:28.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:45:28.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:45:28.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:28.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:45:28.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:28.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:28.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:45:28.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:45:28.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2499 2024-08-25T20:45:28.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2498 2024-08-25T20:45:28.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:28.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:29.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:29.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:45:29.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:29.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:29.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:45:29.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:45:29.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:29.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:29.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:29.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:29.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:29.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:29.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2497 2024-08-25T20:45:29.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2501 2024-08-25T20:45:29.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2500 2024-08-25T20:45:29.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:29.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:30.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:30.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:45:30.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:30.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:30.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:45:30.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:45:30.766 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:45:30.765+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae3409800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:45:31.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2503 2024-08-25T20:45:31.627 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2502 2024-08-25T20:45:31.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:31.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:31.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2505 2024-08-25T20:45:31.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2504 2024-08-25T20:45:31.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:31.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:31.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:31.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:45:31.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:31.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:31.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:45:31.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:45:31.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:31.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:45:31.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:31.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:31.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:45:31.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:45:32.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:32.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:45:32.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:32.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:32.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:45:32.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:45:32.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:32.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:45:32.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:32.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:32.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:45:32.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:45:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:45:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:45:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:45:33.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2506 2024-08-25T20:45:33.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2509 2024-08-25T20:45:33.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:34.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:45:34.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:34.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:34.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:45:34.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:45:34.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:34.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:45:34.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:34.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:34.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:45:34.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:45:34.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:34.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:34.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:34.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:34.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:34.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:45:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:45:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:45:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2508 2024-08-25T20:45:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2507 2024-08-25T20:45:34.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:34.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:35.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2511 2024-08-25T20:45:35.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2510 2024-08-25T20:45:35.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:35.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:35.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2513 2024-08-25T20:45:35.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2512 2024-08-25T20:45:35.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:35.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:36.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2515 2024-08-25T20:45:36.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:45:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:45:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:45:36.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:36.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:45:36.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:36.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:36.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:45:36.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:45:37.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2514 2024-08-25T20:45:37.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:37.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2516 2024-08-25T20:45:37.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:37.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:37.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:45:37.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:37.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:37.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:45:37.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:45:37.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:37.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:45:37.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:37.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:37.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:45:37.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:45:38.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2517 2024-08-25T20:45:38.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:38.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2519 2024-08-25T20:45:38.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2518 2024-08-25T20:45:38.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:38.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:38.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:38.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:45:38.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:38.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:38.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:45:38.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:45:38.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:38.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:45:38.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:38.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:38.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:45:38.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:45:38.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:38.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:45:38.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:38.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:38.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:45:38.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:45:38.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:38.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:45:38.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:38.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:38.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:45:38.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:45:39.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:39.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:39.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:39.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:39.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:39.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:39.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2520 2024-08-25T20:45:39.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2521 2024-08-25T20:45:39.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2522 2024-08-25T20:45:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:39.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:39.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:40.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:40.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:45:40.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:40.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:40.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:45:40.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:45:40.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:40.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:45:40.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:40.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:40.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:45:40.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:45:41.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2523 2024-08-25T20:45:41.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2524 2024-08-25T20:45:41.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:41.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:45:41.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:41.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:41.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:45:41.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:45:41.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:41.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:41.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:45:41.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:41.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:41.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:45:41.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:45:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:45:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:45:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:45:41.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:41.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2525 2024-08-25T20:45:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:42.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2526 2024-08-25T20:45:42.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:45:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:45:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:45:42.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:42.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:45:42.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:42.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:42.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:45:42.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:42.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:45:42.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:45:42.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:42.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:42.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:45:42.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:45:43.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2528 2024-08-25T20:45:43.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2529 2024-08-25T20:45:43.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:43.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2527 2024-08-25T20:45:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:43.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2532 2024-08-25T20:45:43.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:44.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2530 2024-08-25T20:45:44.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2531 2024-08-25T20:45:44.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:44.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:45:44.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:44.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:44.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:45:44.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:45:44.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:44.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:44.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:44.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:44.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:44.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:44.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:44.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:45:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:45:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:45:45.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:45.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:45:45.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:45.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:45.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:45:45.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:45:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2533 2024-08-25T20:45:45.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:46.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2534 2024-08-25T20:45:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2535 2024-08-25T20:45:46.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:46.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:46.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:46.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:45:46.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:46.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:46.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:45:46.710 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:45:46.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:46.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:45:46.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:46.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:46.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:45:46.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:45:46.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:46.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:45:46.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:46.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:46.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:45:46.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:45:46.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:46.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:45:46.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:46.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:46.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:45:46.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:45:47.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:47.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:45:47.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:47.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:47.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:45:47.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:45:47.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2536 2024-08-25T20:45:47.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:47.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:45:47.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:47.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:47.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:45:47.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:45:47.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:47.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2540 2024-08-25T20:45:47.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:48.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:48.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:45:48.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:48.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:48.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:45:48.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:45:48.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2537 2024-08-25T20:45:48.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2539 2024-08-25T20:45:48.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2538 2024-08-25T20:45:48.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:48.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:48.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:49.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:49.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:45:49.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:49.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:49.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:45:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:45:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2542 2024-08-25T20:45:49.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:49.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2541 2024-08-25T20:45:49.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:49.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:50.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:50.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:45:50.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:50.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:50.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:45:50.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:45:50.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:50.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:45:50.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:50.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:50.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:45:50.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:45:51.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2543 2024-08-25T20:45:51.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:51.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:51.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:45:51.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:51.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:51.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:45:51.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:45:51.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:51.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:45:51.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:51.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:51.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:45:51.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:45:51.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:51.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:45:51.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:51.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:51.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:45:51.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:45:52.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2544 2024-08-25T20:45:52.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2547 2024-08-25T20:45:52.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2545 2024-08-25T20:45:52.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2546 2024-08-25T20:45:52.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:52.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:45:52.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:52.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:52.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:45:52.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:45:52.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:52.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:45:52.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:52.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:52.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:45:52.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:45:52.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:52.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:52.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:52.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:53.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2549 2024-08-25T20:45:53.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2550 2024-08-25T20:45:53.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2548 2024-08-25T20:45:53.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:53.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:53.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:54.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:54.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:45:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:45:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:45:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2552 2024-08-25T20:45:54.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2551 2024-08-25T20:45:54.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:54.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:55.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2553 2024-08-25T20:45:55.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:45:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:45:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:45:55.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:55.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:55.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:55.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:55.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:55.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:55.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:55.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:45:55.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:55.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:55.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:45:55.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:45:55.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:55.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:45:55.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:55.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:55.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:45:55.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:45:56.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2556 2024-08-25T20:45:56.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2555 2024-08-25T20:45:56.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:56.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:56.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:56.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:45:56.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:56.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:56.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:45:56.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:45:56.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:56.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:45:56.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:56.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:56.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:45:56.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:45:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:45:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:45:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:45:57.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2554 2024-08-25T20:45:57.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:57.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:57.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:45:57.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:57.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:57.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:45:57.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:45:57.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:57.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:45:57.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:57.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:57.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:45:57.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:45:58.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2557 2024-08-25T20:45:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:45:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:45:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:45:58.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2560 2024-08-25T20:45:58.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:58.823 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:45:58.822+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae4e6e400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:45:58.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:59.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:59.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:45:59.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:59.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:59.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:45:59.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:45:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:45:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:45:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:45:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:45:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:45:59.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:45:59.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2559 2024-08-25T20:45:59.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2558 2024-08-25T20:45:59.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:59.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:45:59.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2561 2024-08-25T20:45:59.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:00.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2562 2024-08-25T20:46:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:46:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:46:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:46:00.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2563 2024-08-25T20:46:01.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:46:01.692+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf64d400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:46:01.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:01.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:01.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:46:01.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:01.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:01.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:46:01.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:46:01.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:01.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:46:01.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:01.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:01.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:46:01.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:46:02.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2564 2024-08-25T20:46:02.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2565 2024-08-25T20:46:02.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:02.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:46:02.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:46:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:46:02.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:02.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:46:02.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:02.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:02.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:46:02.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:46:02.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:02.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:02.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:46:02.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:02.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:02.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:46:02.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:46:02.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:02.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2567 2024-08-25T20:46:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:03.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:46:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:46:03.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:46:03.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2570 2024-08-25T20:46:03.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2569 2024-08-25T20:46:03.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:03.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:04.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:04.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:46:04.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:04.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:04.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:46:04.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:46:04.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2568 2024-08-25T20:46:04.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2566 2024-08-25T20:46:04.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:04.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:04.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2572 2024-08-25T20:46:04.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2571 2024-08-25T20:46:05.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:05.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2573 2024-08-25T20:46:05.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:05.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:05.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:46:05.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:05.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:05.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:46:05.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:46:05.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:05.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:46:05.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:05.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:05.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:46:05.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:46:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:46:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:05.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:46:05.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:46:06.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2574 2024-08-25T20:46:06.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2575 2024-08-25T20:46:06.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:06.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:06.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:06.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:46:06.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:06.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:06.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:46:06.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:46:06.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:06.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:46:06.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:06.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:06.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:46:06.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:46:07.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2576 2024-08-25T20:46:07.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2578 2024-08-25T20:46:07.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:46:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:46:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:46:07.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:07.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:46:07.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:07.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:07.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:46:07.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:46:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:46:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:07.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:46:07.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:46:08.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2577 2024-08-25T20:46:08.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:08.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2579 2024-08-25T20:46:08.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:08.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:08.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:46:08.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:08.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:08.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:46:08.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:46:08.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:08.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:46:08.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:08.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:08.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:46:08.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:46:09.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2581 2024-08-25T20:46:09.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:09.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:46:09.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:09.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:09.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:46:09.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:46:09.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:09.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:46:09.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:46:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:46:09.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:09.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2582 2024-08-25T20:46:09.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:10.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:10.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:46:10.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:10.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:10.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:46:10.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:46:10.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:10.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:46:10.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:10.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:10.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:46:10.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:46:10.699 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:46:10.699+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf63f800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:46:10.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2584 2024-08-25T20:46:10.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:11.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2583 2024-08-25T20:46:11.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2580 2024-08-25T20:46:11.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:11.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:11.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:46:11.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:11.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:11.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:46:11.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:46:11.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:46:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:11.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:46:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:46:11.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2587 2024-08-25T20:46:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:12.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2585 2024-08-25T20:46:12.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:12.701 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:46:12.700+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf51d400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:46:12.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:12.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:46:12.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:12.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:12.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:46:12.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:46:12.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:12.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:46:12.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:12.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:12.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:46:12.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:46:13.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2586 2024-08-25T20:46:13.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:13.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:13.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:46:13.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:13.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:13.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:46:13.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:46:14.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2588 2024-08-25T20:46:14.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2589 2024-08-25T20:46:14.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:14.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:46:14.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:14.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:14.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:46:14.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:46:14.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:14.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:46:14.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:14.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:14.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:46:14.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:46:14.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2591 2024-08-25T20:46:14.706 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:46:14.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:14.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:14.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:46:14.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:46:14.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2592 2024-08-25T20:46:14.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:15.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2590 2024-08-25T20:46:15.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:15.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:46:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:46:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:46:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2595 2024-08-25T20:46:15.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2593 2024-08-25T20:46:15.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2594 2024-08-25T20:46:15.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:15.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:15.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:16.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2596 2024-08-25T20:46:16.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:16.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:16.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:46:16.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:16.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:16.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:46:16.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:46:17.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2597 2024-08-25T20:46:17.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:17.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:17.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:46:17.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:17.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:17.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:46:17.709 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:46:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:17.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:46:17.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:17.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:17.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:46:17.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:46:17.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:17.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:46:17.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:17.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:17.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:46:17.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:46:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:46:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:46:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:46:18.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:18.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:46:18.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:18.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:18.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:46:18.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:46:18.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2598 2024-08-25T20:46:18.704 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2600 2024-08-25T20:46:18.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2601 2024-08-25T20:46:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:46:18.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:46:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:46:18.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:18.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:46:18.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:18.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:18.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:46:18.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:46:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:46:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:46:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:46:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:18.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:18.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:46:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:46:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:46:19.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2599 2024-08-25T20:46:19.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:19.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2603 2024-08-25T20:46:19.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2602 2024-08-25T20:46:19.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:19.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:20.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:20.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:46:20.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:20.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:20.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:46:20.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:46:20.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2604 2024-08-25T20:46:20.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:21.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2605 2024-08-25T20:46:21.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2606 2024-08-25T20:46:21.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:21.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:21.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2607 2024-08-25T20:46:21.712 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:21.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:21.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:46:21.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:21.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:21.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:46:21.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:46:21.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:21.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:46:21.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:21.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:21.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:46:21.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:46:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:46:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:46:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:46:22.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:22.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:46:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:46:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:46:22.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:22.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:46:22.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:22.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:22.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:46:22.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:46:22.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:22.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:46:22.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:22.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:22.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:46:22.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:46:23.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2608 2024-08-25T20:46:23.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:23.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:46:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:46:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:46:23.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:23.815 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:46:23.814+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf621400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:46:23.824 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:46:23.823+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae4c46000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:46:23.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2612 2024-08-25T20:46:23.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:46:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:46:24.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:46:24.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:24.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:46:24.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:24.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:24.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:46:24.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:46:24.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2610 2024-08-25T20:46:24.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2609 2024-08-25T20:46:24.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:24.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:46:24.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:24.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:24.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:46:24.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:46:24.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:24.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:25.705 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2611 2024-08-25T20:46:25.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2613 2024-08-25T20:46:25.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2614 2024-08-25T20:46:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:25.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2616 2024-08-25T20:46:25.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2615 2024-08-25T20:46:25.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:25.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:26.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2617 2024-08-25T20:46:26.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:26.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:26.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:46:26.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:26.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:26.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:46:26.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:46:26.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:26.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:46:26.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:26.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:26.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:46:26.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:46:27.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2619 2024-08-25T20:46:27.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:27.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:46:27.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:27.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:27.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:46:27.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:46:27.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:27.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:46:27.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 106955 2024-08-25T20:46:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2618 2024-08-25T20:46:28.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:28.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:28.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:46:28.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:28.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:28.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:46:28.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:46:28.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:46:28.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 107087 2024-08-25T20:46:28.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2620 2024-08-25T20:46:28.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:28.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:46:28.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:46:28.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:46:28.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:46:28.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:46:28.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:46:28.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:46:28.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 107219 2024-08-25T20:46:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:46:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 107371 2024-08-25T20:46:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:46:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 107524 2024-08-25T20:46:29.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2621 2024-08-25T20:46:29.760 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:46:29.759+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae3417c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:46:29.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:30.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2622 2024-08-25T20:46:30.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:46:31.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:46:31.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 107677 2024-08-25T20:46:32.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:46:32.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 107830 2024-08-25T20:46:32.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:46:32.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 108003 2024-08-25T20:46:32.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:46:32.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 108177 2024-08-25T20:46:33.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:46:33.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:46:33.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 106824 2024-08-25T20:49:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:49:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:49:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:49:02.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 106956 2024-08-25T20:49:03.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:49:03.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:49:03.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:49:03.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 107088 2024-08-25T20:49:04.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:49:04.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:49:04.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:49:04.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 107220 2024-08-25T20:49:05.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:49:05.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:49:05.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:49:05.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 107372 2024-08-25T20:49:06.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:49:06.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:49:06.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:49:06.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 107525 2024-08-25T20:49:07.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:49:07.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:49:07.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:49:07.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 107678 2024-08-25T20:49:08.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:49:08.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:49:08.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:49:08.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 107831 2024-08-25T20:49:09.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:49:09.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:49:09.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:49:09.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 108004 2024-08-25T20:49:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:49:10.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:49:10.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:49:10.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 108178 2024-08-25T20:49:11.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:49:11.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:49:11.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:49:11.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:49:11.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:49:11.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:49:11.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 1 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd0 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary1 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary1 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary1 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 2 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd1 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary2 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary2 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary2 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:49:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:49:11.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 3 2024-08-25T20:49:11.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd2 2024-08-25T20:49:11.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary3 2024-08-25T20:49:11.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary3 2024-08-25T20:49:11.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:49:11.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary3 2024-08-25T20:49:11.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:49:11.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 4 2024-08-25T20:49:11.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:49:11.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd3 2024-08-25T20:49:11.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary4 2024-08-25T20:49:11.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary4 2024-08-25T20:49:11.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:49:11.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary4 2024-08-25T20:49:11.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:49:11.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:49:11.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 5 2024-08-25T20:49:11.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd4 2024-08-25T20:49:11.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary5 2024-08-25T20:49:11.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary5 2024-08-25T20:49:11.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:49:11.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary5 2024-08-25T20:49:11.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:49:11.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:49:11.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 6 2024-08-25T20:49:11.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd5 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary6 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary6 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary6 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 7 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd6 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary7 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary7 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary7 2024-08-25T20:49:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 8 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd7 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary8 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary8 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary8 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 9 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd8 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary9 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary9 2024-08-25T20:49:11.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:49:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary9 2024-08-25T20:49:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:49:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 10 2024-08-25T20:49:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:49:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 114139 2024-08-25T20:49:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd9 2024-08-25T20:49:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary10 2024-08-25T20:49:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary10 2024-08-25T20:49:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:49:11.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary10 2024-08-25T20:49:12.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:49:12.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:49:12.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:49:12.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:49:12.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:49:12.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:49:12.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:49:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:49:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:49:12.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:49:12.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:49:12.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:49:12.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:49:12.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:49:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:49:12.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:49:12.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:49:12.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:49:12.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:49:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:50:05.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=59225af3388ff51d2b8777c84d80be95 2024-08-25T20:50:05.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-08-25T20:50:06.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary5 2024-08-25T20:50:06.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:06.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:06.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:50:06.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-08-25T20:50:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=9c6be675699a8c2a2af757877d56eeb2 2024-08-25T20:50:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-08-25T20:50:06.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary1 2024-08-25T20:50:06.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:06.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:06.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:50:06.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-08-25T20:50:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=5bb98dd80a784152564ed11d422052c2 2024-08-25T20:50:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-08-25T20:50:07.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=2acc994b8752b09ea0dda16b7c998795 2024-08-25T20:50:07.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-08-25T20:50:07.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary7 2024-08-25T20:50:07.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:07.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:07.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:50:07.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-08-25T20:50:07.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary3 2024-08-25T20:50:07.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:07.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:07.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:50:07.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-08-25T20:50:07.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=e0b1d9fe9ca084888ee7d0ebdc88266f 2024-08-25T20:50:07.568 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-08-25T20:50:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=f8f6943eb71a0a339f63069e324ebc34 2024-08-25T20:50:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-08-25T20:50:07.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary6 2024-08-25T20:50:07.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:07.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:07.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:50:07.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-08-25T20:50:07.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary4 2024-08-25T20:50:07.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:07.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:07.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:50:07.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-08-25T20:50:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:50:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-08-25T20:50:07.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:07.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:07.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:50:07.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:07.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:07.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:07.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:07.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:08.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=b1b7fe651890020db22b000f2e358ea7 2024-08-25T20:50:08.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-08-25T20:50:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=37517afb18d9fd254f4267d6e276e2da 2024-08-25T20:50:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-08-25T20:50:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary9 2024-08-25T20:50:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:50:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-08-25T20:50:08.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=8c0daa230f61d9232f93495985fa2705 2024-08-25T20:50:08.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-08-25T20:50:08.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary10 2024-08-25T20:50:08.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:08.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:08.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:50:08.447 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-08-25T20:50:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary8 2024-08-25T20:50:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:08.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:50:08.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-08-25T20:50:08.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:50:08.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:50:08.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:50:08.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:08.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:08.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:50:08.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:08.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:08.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:08.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:08.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-08-25T20:50:08.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:08.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:08.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:50:08.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:08.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:08.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:08.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:08.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:08.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:08.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:50:08.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=3144fc714e38b2911d11a380f84f487f 2024-08-25T20:50:08.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-08-25T20:50:09.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-08-25T20:50:09.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:09.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:09.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:50:09.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:09.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:09.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:09.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:09.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:09.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary2 2024-08-25T20:50:09.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:09.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:09.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:50:09.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-08-25T20:50:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:50:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0cd02d4a-436f-4aa7-aa9d-42409daf184b 2024-08-25T20:50:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:09.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:09.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:09.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:09.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:50:09.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:50:09.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:50:09.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-08-25T20:50:09.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:09.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:09.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:50:09.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:09.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:50:09.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:09.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:09.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:50:09.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:09.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:09.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:09.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:50:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-08-25T20:50:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:50:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:09.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:09.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:10.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:50:10.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:50:10.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:10.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:10.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:50:10.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:10.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:10.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:10.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:10.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:10.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:50:10.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-08-25T20:50:10.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:10.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:10.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:50:10.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:10.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:10.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:10.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:10.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:10.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:50:10.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 3b1376ab-05fd-460d-9be4-66b67ff501ea 2024-08-25T20:50:10.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:10.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:10.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:10.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:10.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:10.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:10.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:10.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:50:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:50:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e92661f6-6b6c-46fb-92be-0c323b12c6ab 2024-08-25T20:50:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:50:10.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:50:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-08-25T20:50:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:50:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:50:10.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:50:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:10.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:10.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:10.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:50:10.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:50:10.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:11.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:11.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:50:11.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:11.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:11.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:11.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:11.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:11.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:50:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:50:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:50:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:11.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:50:11.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:50:11.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 02f58d00-dc8f-41ff-89a9-c8a6ca6f2c72 2024-08-25T20:50:11.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:11.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:11.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:11.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:11.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:11.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:11.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:11.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:50:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:50:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1bee555-24e4-4fa8-9d86-30eac06d0a76 2024-08-25T20:50:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:11.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:11.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:11.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:50:11.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:50:11.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 463c84e4-50f5-4605-b0bd-7e42905093fc 2024-08-25T20:50:11.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:11.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:11.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:11.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:11.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:11.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:11.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:11.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:50:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:50:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:50:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:11.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:11.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:11.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:50:11.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:50:11.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0cd02d4a-436f-4aa7-aa9d-42409daf184b 2024-08-25T20:50:11.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:11.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:11.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:11.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:11.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:11.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:11.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:11.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:50:11.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:50:11.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-08-25T20:50:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:50:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:50:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:50:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:50:12.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:50:12.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8e8e6eb4-4c83-4c1e-842f-1191b4695a8c 2024-08-25T20:50:12.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:12.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:12.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:12.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:12.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:12.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:12.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:12.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:50:12.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:50:12.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:12.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:12.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:50:12.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:12.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:12.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:12.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:12.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:12.220 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:50:12.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:50:12.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:12.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:12.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:50:12.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:12.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:12.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:12.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:12.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:12.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:50:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:50:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 3b1376ab-05fd-460d-9be4-66b67ff501ea 2024-08-25T20:50:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:12.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:12.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:12.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:50:12.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:50:12.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e92661f6-6b6c-46fb-92be-0c323b12c6ab 2024-08-25T20:50:12.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:12.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:12.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:12.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:12.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:12.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:12.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:12.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:50:12.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:50:12.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:50:12.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-08-25T20:50:12.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:12.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:12.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:50:12.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:12.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:12.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:12.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:12.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-08-25T20:50:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:50:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:12.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:12.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:50:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:50:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:12.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:50:12.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:50:12.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 95f0a085-22a4-4bbc-9daf-f1488ce5a207 2024-08-25T20:50:12.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:12.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:12.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:12.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:12.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:12.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:12.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:12.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:50:13.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:50:13.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:13.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:13.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:50:13.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:13.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:50:13.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:50:13.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:13.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:13.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:50:13.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:13.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:50:13.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:50:13.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:13.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:13.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:50:13.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:13.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:13.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:13.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:13.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:13.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:50:13.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:50:13.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 02f58d00-dc8f-41ff-89a9-c8a6ca6f2c72 2024-08-25T20:50:13.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:13.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:13.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:13.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:13.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:13.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:13.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:13.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:50:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:50:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1bee555-24e4-4fa8-9d86-30eac06d0a76 2024-08-25T20:50:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:13.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:13.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:13.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:50:13.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:50:13.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 463c84e4-50f5-4605-b0bd-7e42905093fc 2024-08-25T20:50:13.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:13.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:13.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:13.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:13.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:13.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:13.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:13.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:50:13.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:50:13.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:50:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:50:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:50:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:50:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:13.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:13.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:13.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:13.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:13.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:50:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:50:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416 2024-08-25T20:50:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:13.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:50:13.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 60684371-5dff-4e57-aa1c-760325a2a3a3 2024-08-25T20:50:13.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:13.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:13.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:13.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:13.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:13.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:13.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:13.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:50:13.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:13.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:13.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:50:14.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:50:14.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:14.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:14.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:50:14.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:14.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:14.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:14.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:14.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:14.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:50:14.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:50:14.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8e8e6eb4-4c83-4c1e-842f-1191b4695a8c 2024-08-25T20:50:14.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:14.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:14.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:14.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:14.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:14.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:14.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:14.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:50:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:50:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:50:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:15.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:15.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:50:15.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:50:15.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 95f0a085-22a4-4bbc-9daf-f1488ce5a207 2024-08-25T20:50:15.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:15.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:15.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:15.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:15.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:50:15.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:50:15.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:15.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:15.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:50:15.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:15.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:15.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:15.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:15.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:15.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:50:15.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:50:15.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0cd02d4a-436f-4aa7-aa9d-42409daf184b 2024-08-25T20:50:15.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:15.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:15.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:15.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:15.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:15.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:15.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:15.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:15.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:50:15.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:15.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:15.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:50:15.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:15.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:50:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:50:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:50:15.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:15.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:50:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:50:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 60684371-5dff-4e57-aa1c-760325a2a3a3 2024-08-25T20:50:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:16.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:16.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:50:16.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416 2024-08-25T20:50:16.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:16.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:16.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:16.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:16.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:16.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:16.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:16.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:50:16.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:16.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:16.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:50:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:50:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:50:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:50:16.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:50:16.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:16.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:16.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:50:16.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:16.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:16.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:16.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:16.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:16.341 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:50:16.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:50:16.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 3b1376ab-05fd-460d-9be4-66b67ff501ea 2024-08-25T20:50:16.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:16.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:16.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:16.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:16.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:16.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:16.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:50:16.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e92661f6-6b6c-46fb-92be-0c323b12c6ab 2024-08-25T20:50:16.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:16.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:16.417 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:16.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:16.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:16.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:16.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:16.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:50:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:50:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:17.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:17.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:50:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:50:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:50:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:17.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:17.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:17.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:50:17.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:50:17.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:17.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:17.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:50:17.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:17.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:17.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:17.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:17.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:17.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:50:17.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:50:17.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 02f58d00-dc8f-41ff-89a9-c8a6ca6f2c72 2024-08-25T20:50:17.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:17.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:17.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:17.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:17.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:17.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:17.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:17.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:50:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1bee555-24e4-4fa8-9d86-30eac06d0a76 2024-08-25T20:50:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:17.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:17.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:17.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:50:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 463c84e4-50f5-4605-b0bd-7e42905093fc 2024-08-25T20:50:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:17.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:17.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:17.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:18.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:50:18.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:18.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:18.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:50:18.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:18.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:18.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:18.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:18.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:18.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:50:18.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:50:18.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8e8e6eb4-4c83-4c1e-842f-1191b4695a8c 2024-08-25T20:50:18.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:18.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:18.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:18.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:18.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:18.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:18.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:18.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:19.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:50:19.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:19.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:19.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:50:19.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:19.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:19.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:19.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:19.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:19.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:50:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:50:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 95f0a085-22a4-4bbc-9daf-f1488ce5a207 2024-08-25T20:50:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:19.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:19.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:19.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:19.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:20.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:50:20.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:20.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:20.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:50:20.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:20.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:20.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:20.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:20.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:20.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:50:20.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:50:20.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:20.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:20.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:50:20.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:20.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:20.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:20.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:20.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:20.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:50:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:50:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 60684371-5dff-4e57-aa1c-760325a2a3a3 2024-08-25T20:50:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:50:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416 2024-08-25T20:50:20.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:20.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:20.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:20.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:20.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:20.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:20.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:20.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:20.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:20.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:20.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:23.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:50:23.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:23.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:23.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:50:23.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:23.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:23.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:23.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:23.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:23.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:50:23.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:50:23.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0cd02d4a-436f-4aa7-aa9d-42409daf184b 2024-08-25T20:50:23.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:23.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:23.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:23.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:23.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:23.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:23.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:23.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:50:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:50:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:24.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:24.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:50:24.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:50:24.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:24.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:24.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:50:24.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:24.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:24.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:24.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:24.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:24.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:50:24.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:50:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 3b1376ab-05fd-460d-9be4-66b67ff501ea 2024-08-25T20:50:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:24.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:24.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:24.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:24.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:50:24.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e92661f6-6b6c-46fb-92be-0c323b12c6ab 2024-08-25T20:50:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:25.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:50:25.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:50:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:25.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:25.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:25.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:50:25.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:50:25.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:25.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:25.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:50:25.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:25.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:25.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:25.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:25.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:25.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:25.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:25.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:50:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:50:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 02f58d00-dc8f-41ff-89a9-c8a6ca6f2c72 2024-08-25T20:50:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:25.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:50:25.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1bee555-24e4-4fa8-9d86-30eac06d0a76 2024-08-25T20:50:25.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:25.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:25.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:25.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:25.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:25.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:25.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:25.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:25.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:25.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:25.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:25.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:50:25.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 463c84e4-50f5-4605-b0bd-7e42905093fc 2024-08-25T20:50:25.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:25.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:25.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:25.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:25.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:25.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:25.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:25.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:26.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:50:26.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:26.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:26.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:50:26.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:26.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:26.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:26.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:26.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:26.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:50:26.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:50:26.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8e8e6eb4-4c83-4c1e-842f-1191b4695a8c 2024-08-25T20:50:26.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:26.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:26.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:26.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:26.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:26.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:26.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:26.275 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:50:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:50:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:27.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:27.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:50:27.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:50:27.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 95f0a085-22a4-4bbc-9daf-f1488ce5a207 2024-08-25T20:50:27.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:27.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:27.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:27.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:27.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:27.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:27.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:27.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:28.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:50:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:50:28.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:50:28.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 60684371-5dff-4e57-aa1c-760325a2a3a3 2024-08-25T20:50:28.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:28.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:28.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:28.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:28.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:28.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:50:28.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416 2024-08-25T20:50:28.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:50:28.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:50:28.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:28.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:49:58' 2024-08-25T20:50:28.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:28.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:28.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:50:28.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:28.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:28.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:28.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:50:31.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:50:31.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:31.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:31.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:50:31.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:31.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:31.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:31.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:31.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:31.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:50:31.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:50:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0cd02d4a-436f-4aa7-aa9d-42409daf184b 2024-08-25T20:50:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:31.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:31.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:31.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:31.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:31.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:31.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:31.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:31.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:31.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:31.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:31.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:31.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:31.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:31.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:31.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:31.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:50:31.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:31.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:31.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:31.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:50:31.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:31.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:50:31.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:31.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:31.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:31.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:31.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:31.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:31.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-08-25T20:50:31.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:31.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:31.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:50:31.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:31.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:31.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:31.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:31.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:32.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:50:32.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:32.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:32.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:50:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:32.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:50:32.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:50:32.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:32.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:32.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:50:32.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:32.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:32.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:32.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:32.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:32.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:50:32.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:50:32.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 3b1376ab-05fd-460d-9be4-66b67ff501ea 2024-08-25T20:50:32.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:32.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:32.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:32.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:32.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:32.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:32.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:32.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:32.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:32.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:32.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:32.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:32.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:32.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:32.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:32.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:32.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:32.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:32.563 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:32.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:32.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:50:32.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:32.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:32.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:32.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:32.564 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:50:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e92661f6-6b6c-46fb-92be-0c323b12c6ab 2024-08-25T20:50:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:32.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:32.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:32.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:32.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:32.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:32.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:32.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:32.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:32.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:32.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:32.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:32.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:32.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:32.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:32.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:32.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:32.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:32.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:50:32.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:32.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:32.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:32.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:32.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:32.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:32.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:32.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:50:32.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:32.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:50:32.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:32.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:32.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:32.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:32.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:32.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:32.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-08-25T20:50:32.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:32.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:32.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:50:32.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:32.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:32.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:32.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:32.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:32.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:32.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:32.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:50:32.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:32.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:50:32.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:32.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:32.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:32.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-08-25T20:50:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:50:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:32.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-08-25T20:50:32.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:32.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:32.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:50:32.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:32.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:32.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:32.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:32.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:32.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-08-25T20:50:33.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:50:33.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0cd02d4a-436f-4aa7-aa9d-42409daf184b 2024-08-25T20:50:33.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:33.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:33.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:33.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:33.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:33.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:33.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:33.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:33.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:33.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:33.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:33.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:33.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:33.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:33.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:50:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:33.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:33.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:33.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:33.179 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:50:33.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:33.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:50:33.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:33.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:33.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary1 2024-08-25T20:50:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:50:33.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:50:33.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-08-25T20:50:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:50:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:50:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:50:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:50:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:50:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:50:33.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:50:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:50:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:33.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:50:33.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:50:33.464 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 02f58d00-dc8f-41ff-89a9-c8a6ca6f2c72 2024-08-25T20:50:33.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:33.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:33.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:33.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:33.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:33.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:50:33.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1bee555-24e4-4fa8-9d86-30eac06d0a76 2024-08-25T20:50:33.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:33.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:33.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:33.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:33.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:33.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:33.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:33.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:33.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:33.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:33.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:33.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:33.470 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:33.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:33.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:33.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:33.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:33.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:33.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:33.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:33.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:33.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:33.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:33.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:33.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:33.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:33.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:33.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:50:33.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:33.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:33.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:33.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:33.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:50:33.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:33.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:33.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:33.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:33.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:33.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:33.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:50:33.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 463c84e4-50f5-4605-b0bd-7e42905093fc 2024-08-25T20:50:33.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:33.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:33.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:33.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:33.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:33.487 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:33.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:33.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:33.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:33.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:33.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:33.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:33.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:33.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:33.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:33.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:33.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:33.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:33.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:50:33.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:33.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:33.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:33.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:33.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:33.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:33.626 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:50:33.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:33.630 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:33.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:50:33.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:33.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:50:33.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:33.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:50:33.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:33.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:33.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:33.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-08-25T20:50:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:50:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:50:33.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-08-25T20:50:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:50:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:33.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:33.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:33.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:50:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:50:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:50:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:50:33.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-08-25T20:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:33.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:33.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:33.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:33.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-08-25T20:50:33.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:33.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:50:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:33.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-08-25T20:50:33.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:33.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:50:33.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:33.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:33.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:33.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:33.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-08-25T20:50:33.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-08-25T20:50:33.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:33.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:50:33.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:33.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:33.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:33.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:33.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-08-25T20:50:33.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:50:33.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 3b1376ab-05fd-460d-9be4-66b67ff501ea 2024-08-25T20:50:33.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:33.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:33.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:33.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:33.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:33.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:33.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:33.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:33.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:33.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:33.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:33.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:33.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:33.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:33.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:33.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:50:33.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:33.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:33.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:33.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:33.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:50:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e92661f6-6b6c-46fb-92be-0c323b12c6ab 2024-08-25T20:50:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd0 2024-08-25T20:50:33.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:50:33.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:50:33.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:33.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:33.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:33.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:33.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:33.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:33.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:33.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:33.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:50:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:33.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:33.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:33.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:50:33.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:33.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:50:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:33.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:33.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:33.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:33.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:33.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary3 2024-08-25T20:50:33.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:33.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:50:33.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:50:33.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-08-25T20:50:33.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:33.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:33.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:50:33.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:33.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:50:33.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:33.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:33.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:33.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary7 2024-08-25T20:50:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:50:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:50:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-08-25T20:50:34.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:50:34.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:34.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:34.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:50:34.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:34.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:34.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:50:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:50:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8e8e6eb4-4c83-4c1e-842f-1191b4695a8c 2024-08-25T20:50:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:34.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:34.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:34.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:34.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:34.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:34.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:34.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:34.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:34.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:34.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:34.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:34.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:34.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:34.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:34.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:34.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:34.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:50:34.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:34.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:34.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:34.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:34.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:34.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:34.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:34.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:50:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:34.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:50:34.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:34.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:34.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:34.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-08-25T20:50:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:50:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:34.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:34.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:34.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:50:34.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-08-25T20:50:34.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:34.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:34.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:50:34.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:34.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:34.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:34.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:34.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:34.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-08-25T20:50:34.646 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:50:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-08-25T20:50:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:50:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:34.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:34.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-08-25T20:50:34.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:50:34.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:50:34.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:50:34.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:50:34.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-08-25T20:50:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-08-25T20:50:34.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-08-25T20:50:34.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:34.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:34.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:50:34.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:34.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:34.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:34.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:34.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-08-25T20:50:34.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:50:34.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 02f58d00-dc8f-41ff-89a9-c8a6ca6f2c72 2024-08-25T20:50:34.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:34.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:34.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:34.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:34.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:34.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:50:34.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1bee555-24e4-4fa8-9d86-30eac06d0a76 2024-08-25T20:50:34.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:34.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:34.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:34.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:34.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:34.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:34.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:34.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:34.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:34.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:34.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:34.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:34.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:34.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:34.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:34.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:34.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:34.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:34.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:50:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:50:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:34.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:34.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:34.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:34.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:34.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:34.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:50:34.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 463c84e4-50f5-4605-b0bd-7e42905093fc 2024-08-25T20:50:34.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:34.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:34.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:34.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:34.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:34.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:34.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:34.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:34.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:34.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:34.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:34.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:34.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:34.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:34.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:34.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:34.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:34.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:34.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:50:34.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:34.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:34.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:34.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:34.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:34.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd1 2024-08-25T20:50:34.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:50:34.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:50:34.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd2 2024-08-25T20:50:34.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:50:34.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:50:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:50:35.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:35.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:35.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:50:35.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:50:35.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:35.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:35.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:50:35.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:35.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary6 2024-08-25T20:50:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:35.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:35.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:50:35.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:50:35.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-08-25T20:50:35.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:35.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:35.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:35.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:35.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary4 2024-08-25T20:50:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:50:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:50:35.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-08-25T20:50:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:50:35.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:35.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:50:35.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:35.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:35.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:35.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:35.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:35.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:35.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary2 2024-08-25T20:50:35.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:35.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:35.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:50:35.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:50:35.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-08-25T20:50:35.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:50:35.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:50:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:35.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:50:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:50:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 95f0a085-22a4-4bbc-9daf-f1488ce5a207 2024-08-25T20:50:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:35.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:35.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:35.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:35.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:35.349 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:35.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:35.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:35.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:35.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:35.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:35.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:35.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:50:35.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:35.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:35.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:35.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:35.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:35.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:35.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:35.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:50:35.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-08-25T20:50:35.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:35.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:35.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:50:35.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:35.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:35.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:35.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:35.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:35.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-08-25T20:50:35.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:35.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:50:35.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:35.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:35.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:35.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:35.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:35.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:35.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-08-25T20:50:35.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:35.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:35.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:50:35.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:35.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:35.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:35.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:35.500 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:35.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:50:35.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8e8e6eb4-4c83-4c1e-842f-1191b4695a8c 2024-08-25T20:50:35.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:35.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:35.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:35.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:35.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:35.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:35.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:35.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:35.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:35.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:35.574 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:35.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:35.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:35.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:35.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:35.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:35.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:35.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:35.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:35.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:35.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:50:35.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:35.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:35.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:35.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:35.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:35.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:50:35.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:50:35.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:50:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:50:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:50:35.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-08-25T20:50:35.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:50:35.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-08-25T20:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:50:35.725 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-08-25T20:50:35.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:35.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:35.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:50:35.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:35.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:50:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:35.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:35.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:35.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:35.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:35.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:35.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary5 2024-08-25T20:50:35.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:35.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:35.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:50:35.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:50:35.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-08-25T20:50:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd3 2024-08-25T20:50:35.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:50:35.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:50:35.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd4 2024-08-25T20:50:35.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:50:35.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:50:35.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd5 2024-08-25T20:50:35.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:50:35.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:50:36.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:50:36.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:50:36.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:36.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:36.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:50:36.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:36.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:36.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:36.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:36.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:36.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:50:36.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:36.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:36.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:36.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:36.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:36.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:36.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:50:36.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:36.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:50:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:50:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 60684371-5dff-4e57-aa1c-760325a2a3a3 2024-08-25T20:50:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:50:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416 2024-08-25T20:50:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:36.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:36.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:36.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:36.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:36.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:36.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:36.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:36.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:36.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:36.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:36.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:36.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:36.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:36.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:36.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:36.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:36.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:36.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:36.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:36.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:36.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:36.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:36.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:36.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:50:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:36.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:36.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:50:36.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:36.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:50:36.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:36.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:36.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:36.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:36.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:36.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:36.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-08-25T20:50:36.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:36.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:36.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:50:36.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:36.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:36.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:36.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:36.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:36.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-08-25T20:50:36.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:36.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:36.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:50:36.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:36.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:50:36.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:36.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:36.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:36.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:36.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:36.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:50:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:36.612 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:36.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:36.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-08-25T20:50:36.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:50:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:36.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:36.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:50:36.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:36.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:36.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-08-25T20:50:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:50:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:50:36.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:50:36.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:50:36.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 95f0a085-22a4-4bbc-9daf-f1488ce5a207 2024-08-25T20:50:36.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:36.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:36.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:36.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:36.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:36.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:36.696 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:36.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:36.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:36.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:36.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:36.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:36.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:36.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:36.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:50:36.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:36.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:36.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:50:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:36.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:50:36.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:50:36.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-08-25T20:50:36.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:36.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:36.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:50:36.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:36.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:50:37.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:37.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:37.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:37.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:37.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:37.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:37.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary10 2024-08-25T20:50:37.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:37.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:37.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:50:37.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:50:37.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-08-25T20:50:37.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd6 2024-08-25T20:50:37.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:50:37.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:50:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-08-25T20:50:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:37.623 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:50:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:37.624 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:37.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:37.625 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-08-25T20:50:37.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-08-25T20:50:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:37.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:50:37.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:37.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:50:37.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:50:37.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:37.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:37.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-08-25T20:50:37.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:50:37.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:50:37.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:50:37.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-08-25T20:50:37.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:50:37.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 60684371-5dff-4e57-aa1c-760325a2a3a3 2024-08-25T20:50:37.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:37.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:37.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:37.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:37.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:37.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:37.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:37.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:37.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:37.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:37.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:37.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:50:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416 2024-08-25T20:50:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:50:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:50:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:50:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:50:28' 2024-08-25T20:50:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:50:37.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:37.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:50:37.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:37.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:50:37.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:37.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:37.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:50:37.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:50:37.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:37.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:50:37.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:37.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:50:37.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:37.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:37.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:37.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:37.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:50:37.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:37.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:37.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:37.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:50:37.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:37.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:37.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:50:37.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:50:37.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:37.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-08-25 20:50:28' 2024-08-25T20:50:37.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:37.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:37.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:50:37.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:50:37.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:50:28' 2024-08-25T20:50:37.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:50:37.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:50:37.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:50:38.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd7 2024-08-25T20:50:38.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:50:38.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:50:38.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:38.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:38.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:50:38.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:38.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:50:28' 2024-08-25T20:50:38.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:50:38.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:50:38.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:50:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:50:28' 2024-08-25T20:50:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:38.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:50:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:38.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:38.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:38.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:38.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary9 2024-08-25T20:50:38.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:38.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:38.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:50:38.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:50:38.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-08-25T20:50:38.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:50:38.476 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:50:38.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:50:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:50:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:50:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary8 2024-08-25T20:50:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:50:38.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:50:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:50:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:50:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-08-25T20:50:39.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:50:39.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:50:39.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:50:39.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:50:39.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-08-25T20:50:39.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:50:39.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:50:39.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-08-25T20:50:40.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd8 2024-08-25T20:50:40.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:50:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:50:40.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd9 2024-08-25T20:50:40.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:50:40.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:51:24.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=9c6be675699a8c2a2af757877d56eeb2 2024-08-25T20:51:24.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-08-25T20:51:25.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 9c6be675699a8c2a2af757877d56eeb2 != \9\c\6\b\e\6\7\5\6\9\9\a\8\c\2\a\2\a\f\7\5\7\8\7\7\d\5\6\e\e\b\2 ]] 2024-08-25T20:51:25.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:51:25.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 114140 2024-08-25T20:51:28.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=3144fc714e38b2911d11a380f84f487f 2024-08-25T20:51:28.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-08-25T20:51:29.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 3144fc714e38b2911d11a380f84f487f != \3\1\4\4\f\c\7\1\4\e\3\8\b\2\9\1\1\d\1\1\a\3\8\0\f\8\4\f\4\8\7\f ]] 2024-08-25T20:51:29.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:51:29.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 114142 2024-08-25T20:51:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=5bb98dd80a784152564ed11d422052c2 2024-08-25T20:51:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-08-25T20:51:29.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=f8f6943eb71a0a339f63069e324ebc34 2024-08-25T20:51:29.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-08-25T20:51:29.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 5bb98dd80a784152564ed11d422052c2 != \5\b\b\9\8\d\d\8\0\a\7\8\4\1\5\2\5\6\4\e\d\1\1\d\4\2\2\0\5\2\c\2 ]] 2024-08-25T20:51:29.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ f8f6943eb71a0a339f63069e324ebc34 != \f\8\f\6\9\4\3\e\b\7\1\a\0\a\3\3\9\f\6\3\0\6\9\e\3\2\4\e\b\c\3\4 ]] 2024-08-25T20:51:29.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=2acc994b8752b09ea0dda16b7c998795 2024-08-25T20:51:29.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-08-25T20:51:30.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 2acc994b8752b09ea0dda16b7c998795 != \2\a\c\c\9\9\4\b\8\7\5\2\b\0\9\e\a\0\d\d\a\1\6\b\7\c\9\9\8\7\9\5 ]] 2024-08-25T20:51:30.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:51:30.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 114144 2024-08-25T20:51:30.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:51:30.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 114146 2024-08-25T20:51:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=e0b1d9fe9ca084888ee7d0ebdc88266f 2024-08-25T20:51:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-08-25T20:51:31.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=59225af3388ff51d2b8777c84d80be95 2024-08-25T20:51:31.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-08-25T20:51:31.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ e0b1d9fe9ca084888ee7d0ebdc88266f != \e\0\b\1\d\9\f\e\9\c\a\0\8\4\8\8\8\e\e\7\d\0\e\b\d\c\8\8\2\6\6\f ]] 2024-08-25T20:51:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 59225af3388ff51d2b8777c84d80be95 != \5\9\2\2\5\a\f\3\3\8\8\f\f\5\1\d\2\b\8\7\7\7\c\8\4\d\8\0\b\e\9\5 ]] 2024-08-25T20:51:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:51:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 114148 2024-08-25T20:51:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:51:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 114150 2024-08-25T20:51:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:51:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 114152 2024-08-25T20:51:31.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=37517afb18d9fd254f4267d6e276e2da 2024-08-25T20:51:31.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-08-25T20:51:31.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 37517afb18d9fd254f4267d6e276e2da != \3\7\5\1\7\a\f\b\1\8\d\9\f\d\2\5\4\f\4\2\6\7\d\6\e\2\7\6\e\2\d\a ]] 2024-08-25T20:51:32.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=b1b7fe651890020db22b000f2e358ea7 2024-08-25T20:51:32.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-08-25T20:51:32.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ b1b7fe651890020db22b000f2e358ea7 != \b\1\b\7\f\e\6\5\1\8\9\0\0\2\0\d\b\2\2\b\0\0\0\f\2\e\3\5\8\e\a\7 ]] 2024-08-25T20:51:33.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=8c0daa230f61d9232f93495985fa2705 2024-08-25T20:51:33.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-08-25T20:51:33.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 8c0daa230f61d9232f93495985fa2705 != \8\c\0\d\a\a\2\3\0\f\6\1\d\9\2\3\2\f\9\3\4\9\5\9\8\5\f\a\2\7\0\5 ]] 2024-08-25T20:51:33.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:51:33.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 114154 2024-08-25T20:51:33.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:51:33.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 114155 2024-08-25T20:51:33.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:51:33.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:51:33.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:33.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:51:34.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-08-25T20:51:34.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:34.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:34.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:51:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:34.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:51:34.477 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:51:34.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0cd02d4a-436f-4aa7-aa9d-42409daf184b 2024-08-25T20:51:34.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:51:34.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3215155.2,"bytes_per_snapshot":64303104.0,"last_snapshot_bytes":64303104,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619033,"remote_snapshot_timestamp":1724619033,"replay_state":"idle"} 2024-08-25T20:51:34.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:51:34.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:51:28' 2024-08-25T20:51:34.478 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:51:34.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:51:34.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:51:34.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3215155.2,"bytes_per_snapshot":64303104.0,"last_snapshot_bytes":64303104,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619033,"remote_snapshot_timestamp":1724619033,"replay_state":"idle"} 2024-08-25T20:51:34.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:51:34.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:34.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:51:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:51:34.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:51:34.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:51:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:51:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-08-25 20:51:28' 2024-08-25T20:51:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:51:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:51:28' 2024-08-25T20:51:34.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:34.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:34.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:51:34.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10cd02d4a-436f-4aa7-aa9d-42409daf184bup+replayingreplaying, {"bytes_per_second":3215155.2,"bytes_per_snapshot":64303104.0,"last_snapshot_bytes":64303104,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619033,"remote_snapshot_timestamp":1724619033,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+replayingreplaying, {"bytes_per_second":3633971.2,"bytes_per_snapshot":72679424.0,"last_snapshot_bytes":72679424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619037,"remote_snapshot_timestamp":1724619037,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+replayingreplaying, {"bytes_per_second":3232358.4,"bytes_per_snapshot":64647168.0,"last_snapshot_bytes":64647168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+replayingreplaying, {"bytes_per_second":3422208.0,"bytes_per_snapshot":68444160.0,"last_snapshot_bytes":68444160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+replayingreplaying, {"bytes_per_second":3149414.4,"bytes_per_snapshot":62988288.0,"last_snapshot_bytes":62988288,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+replayingreplaying, {"bytes_per_second":3460710.4,"bytes_per_snapshot":69214208.0,"last_snapshot_bytes":69214208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619036,"remote_snapshot_timestamp":1724619036,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+replayingreplaying, {"bytes_per_second":3203891.2,"bytes_per_snapshot":64077824.0,"last_snapshot_bytes":64077824,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+replayingreplaying, {"bytes_per_second":3327795.2,"bytes_per_snapshot":66555904.0,"last_snapshot_bytes":66555904,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+replayingreplaying, {"bytes_per_second":3205529.6,"bytes_per_snapshot":64110592.0,"last_snapshot_bytes":64110592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+replayingreplaying, {"bytes_per_second":3378380.8,"bytes_per_snapshot":67567616.0,"last_snapshot_bytes":67567616,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28' 2024-08-25T20:51:34.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:51:34.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:51:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:34.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:51:34.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:51:34.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:34.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:51:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:34.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary1 2024-08-25T20:51:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:51:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:51:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-08-25T20:51:34.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-08-25T20:51:34.864 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:34.863+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adfb66ea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:51:34.867 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:34.863+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972780 [3/0cd02d4a-436f-4aa7-aa9d-42409daf184b] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:51:36.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:51:36.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-08-25T20:51:36.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:36.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:36.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:51:36.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:36.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:51:36.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary1 2024-08-25T20:51:36.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:36.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary1 2024-08-25T20:51:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:51:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:51:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-08-25T20:51:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:51:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:51:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-08-25T20:51:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:51:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:51:37.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-08-25T20:51:37.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:37.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:37.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:51:37.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:37.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:37.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:37.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:37.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:37.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:51:38.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:51:38.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 463c84e4-50f5-4605-b0bd-7e42905093fc 2024-08-25T20:51:38.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:51:38.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3232358.4,"bytes_per_snapshot":64647168.0,"last_snapshot_bytes":64647168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"} 2024-08-25T20:51:38.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:51:38.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:51:28' 2024-08-25T20:51:38.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:51:38.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:51:38.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:51:38.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3232358.4,"bytes_per_snapshot":64647168.0,"last_snapshot_bytes":64647168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"} 2024-08-25T20:51:38.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:51:38.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:38.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:51:38.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:38.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:51:38.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:51:38.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:51:38.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:51:38.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:38.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-08-25 20:51:28' 2024-08-25T20:51:38.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:51:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:51:28' 2024-08-25T20:51:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:51:38.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING941904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+replayingreplaying, {"bytes_per_second":3633971.2,"bytes_per_snapshot":72679424.0,"last_snapshot_bytes":72679424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619037,"remote_snapshot_timestamp":1724619037,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary2463c84e4-50f5-4605-b0bd-7e42905093fcup+replayingreplaying, {"bytes_per_second":3232358.4,"bytes_per_snapshot":64647168.0,"last_snapshot_bytes":64647168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+replayingreplaying, {"bytes_per_second":3422208.0,"bytes_per_snapshot":68444160.0,"last_snapshot_bytes":68444160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+replayingreplaying, {"bytes_per_second":3149414.4,"bytes_per_snapshot":62988288.0,"last_snapshot_bytes":62988288,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+replayingreplaying, {"bytes_per_second":3460710.4,"bytes_per_snapshot":69214208.0,"last_snapshot_bytes":69214208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619036,"remote_snapshot_timestamp":1724619036,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+replayingreplaying, {"bytes_per_second":3203891.2,"bytes_per_snapshot":64077824.0,"last_snapshot_bytes":64077824,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+replayingreplaying, {"bytes_per_second":3327795.2,"bytes_per_snapshot":66555904.0,"last_snapshot_bytes":66555904,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+replayingreplaying, {"bytes_per_second":3205529.6,"bytes_per_snapshot":64110592.0,"last_snapshot_bytes":64110592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+replayingreplaying, {"bytes_per_second":3378380.8,"bytes_per_snapshot":67567616.0,"last_snapshot_bytes":67567616,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28' 2024-08-25T20:51:38.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:51:38.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:51:38.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:38.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:51:38.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:51:38.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:38.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:51:38.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary2 2024-08-25T20:51:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:51:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:51:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-08-25T20:51:38.277 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-08-25T20:51:38.421 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:38.421+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4b94b60 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:51:38.421 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:38.421+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ae365f900 [3/463c84e4-50f5-4605-b0bd-7e42905093fc] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:51:39.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:51:39.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-08-25T20:51:39.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:39.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:39.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:51:39.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:39.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:51:39.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary2 2024-08-25T20:51:39.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:39.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary2 2024-08-25T20:51:39.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:51:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:51:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-08-25T20:51:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:51:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:51:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-08-25T20:51:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:51:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:39.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:39.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:39.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:51:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-08-25T20:51:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:51:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:40.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:40.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:51:41.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:51:41.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 3b1376ab-05fd-460d-9be4-66b67ff501ea 2024-08-25T20:51:41.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:51:41.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3422208.0,"bytes_per_snapshot":68444160.0,"last_snapshot_bytes":68444160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"} 2024-08-25T20:51:41.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:51:41.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:51:28' 2024-08-25T20:51:41.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:51:41.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:51:41.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:51:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3422208.0,"bytes_per_snapshot":68444160.0,"last_snapshot_bytes":68444160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"} 2024-08-25T20:51:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:51:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:51:41.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:51:41.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:51:41.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:51:41.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:51:41.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:41.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-08-25 20:51:28' 2024-08-25T20:51:41.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:41.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:41.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:51:41.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:41.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:51:28' 2024-08-25T20:51:41.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:41.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:41.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:51:41.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING841904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+replayingreplaying, {"bytes_per_second":3633971.2,"bytes_per_snapshot":72679424.0,"last_snapshot_bytes":72679424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619037,"remote_snapshot_timestamp":1724619037,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary33b1376ab-05fd-460d-9be4-66b67ff501eaup+replayingreplaying, {"bytes_per_second":3422208.0,"bytes_per_snapshot":68444160.0,"last_snapshot_bytes":68444160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+replayingreplaying, {"bytes_per_second":3149414.4,"bytes_per_snapshot":62988288.0,"last_snapshot_bytes":62988288,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+replayingreplaying, {"bytes_per_second":3460710.4,"bytes_per_snapshot":69214208.0,"last_snapshot_bytes":69214208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619036,"remote_snapshot_timestamp":1724619036,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+replayingreplaying, {"bytes_per_second":3203891.2,"bytes_per_snapshot":64077824.0,"last_snapshot_bytes":64077824,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+replayingreplaying, {"bytes_per_second":3327795.2,"bytes_per_snapshot":66555904.0,"last_snapshot_bytes":66555904,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+replayingreplaying, {"bytes_per_second":3205529.6,"bytes_per_snapshot":64110592.0,"last_snapshot_bytes":64110592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+replayingreplaying, {"bytes_per_second":3378380.8,"bytes_per_snapshot":67567616.0,"last_snapshot_bytes":67567616,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28' 2024-08-25T20:51:41.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:51:41.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:51:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:41.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:51:41.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:51:41.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:41.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:51:41.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:41.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:41.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:41.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary3 2024-08-25T20:51:41.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:51:41.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:41.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:51:41.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-08-25T20:51:41.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-08-25T20:51:41.411 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:41.410+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae36ffa00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:51:41.411 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:41.410+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ae365e000 [3/3b1376ab-05fd-460d-9be4-66b67ff501ea] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:51:42.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:51:42.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-08-25T20:51:42.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:42.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:42.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:51:42.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:42.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:51:42.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary3 2024-08-25T20:51:42.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:42.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary3 2024-08-25T20:51:43.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:51:43.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:51:43.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-08-25T20:51:43.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:43.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:43.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:51:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:51:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-08-25T20:51:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:51:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:51:44.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-08-25T20:51:44.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:44.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:44.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:51:44.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:44.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:44.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:44.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:44.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:51:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:51:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c1bee555-24e4-4fa8-9d86-30eac06d0a76 2024-08-25T20:51:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:51:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3149414.4,"bytes_per_snapshot":62988288.0,"last_snapshot_bytes":62988288,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"} 2024-08-25T20:51:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:51:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:51:28' 2024-08-25T20:51:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:51:44.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:51:44.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:51:44.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3149414.4,"bytes_per_snapshot":62988288.0,"last_snapshot_bytes":62988288,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"} 2024-08-25T20:51:44.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:51:44.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:44.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:51:44.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:44.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:51:44.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:51:44.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:51:44.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:51:44.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:44.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-08-25 20:51:28' 2024-08-25T20:51:44.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:44.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:44.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:51:44.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:44.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:51:28' 2024-08-25T20:51:44.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:44.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:44.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:51:44.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING741904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+replayingreplaying, {"bytes_per_second":3633971.2,"bytes_per_snapshot":72679424.0,"last_snapshot_bytes":72679424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619037,"remote_snapshot_timestamp":1724619037,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary4c1bee555-24e4-4fa8-9d86-30eac06d0a76up+replayingreplaying, {"bytes_per_second":3149414.4,"bytes_per_snapshot":62988288.0,"last_snapshot_bytes":62988288,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+replayingreplaying, {"bytes_per_second":3460710.4,"bytes_per_snapshot":69214208.0,"last_snapshot_bytes":69214208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619036,"remote_snapshot_timestamp":1724619036,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+replayingreplaying, {"bytes_per_second":3203891.2,"bytes_per_snapshot":64077824.0,"last_snapshot_bytes":64077824,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+replayingreplaying, {"bytes_per_second":3327795.2,"bytes_per_snapshot":66555904.0,"last_snapshot_bytes":66555904,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+replayingreplaying, {"bytes_per_second":3205529.6,"bytes_per_snapshot":64110592.0,"last_snapshot_bytes":64110592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+replayingreplaying, {"bytes_per_second":3378380.8,"bytes_per_snapshot":67567616.0,"last_snapshot_bytes":67567616,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28' 2024-08-25T20:51:44.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:51:44.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:51:44.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:44.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:51:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:51:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:51:44.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:44.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:44.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:44.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary4 2024-08-25T20:51:44.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:51:44.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:44.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:51:44.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-08-25T20:51:44.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-08-25T20:51:44.399 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:44.398+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae36fe680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:51:44.399 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:44.398+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972c80 [3/c1bee555-24e4-4fa8-9d86-30eac06d0a76] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:51:45.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:51:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-08-25T20:51:45.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:45.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:45.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:51:45.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:45.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:51:45.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary4 2024-08-25T20:51:45.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:45.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary4 2024-08-25T20:51:45.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:45.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:45.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:51:46.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-08-25T20:51:46.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:46.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:46.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:51:46.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:46.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:46.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:46.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:46.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:46.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:51:46.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:51:46.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 8e8e6eb4-4c83-4c1e-842f-1191b4695a8c 2024-08-25T20:51:46.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:51:46.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3460710.4,"bytes_per_snapshot":69214208.0,"last_snapshot_bytes":69214208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619036,"remote_snapshot_timestamp":1724619036,"replay_state":"idle"} 2024-08-25T20:51:46.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:51:46.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:51:28' 2024-08-25T20:51:46.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:51:46.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:51:46.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:51:46.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3460710.4,"bytes_per_snapshot":69214208.0,"last_snapshot_bytes":69214208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619036,"remote_snapshot_timestamp":1724619036,"replay_state":"idle"} 2024-08-25T20:51:46.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:51:46.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:46.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:51:46.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:46.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:51:46.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:51:46.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:51:46.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:51:46.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:46.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-08-25 20:51:28' 2024-08-25T20:51:46.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:46.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:46.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:51:46.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:46.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:51:28' 2024-08-25T20:51:46.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:46.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:46.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:51:47.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING641904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+replayingreplaying, {"bytes_per_second":3633971.2,"bytes_per_snapshot":72679424.0,"last_snapshot_bytes":72679424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619037,"remote_snapshot_timestamp":1724619037,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary58e8e6eb4-4c83-4c1e-842f-1191b4695a8cup+replayingreplaying, {"bytes_per_second":3460710.4,"bytes_per_snapshot":69214208.0,"last_snapshot_bytes":69214208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619036,"remote_snapshot_timestamp":1724619036,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+replayingreplaying, {"bytes_per_second":3203891.2,"bytes_per_snapshot":64077824.0,"last_snapshot_bytes":64077824,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+replayingreplaying, {"bytes_per_second":3327795.2,"bytes_per_snapshot":66555904.0,"last_snapshot_bytes":66555904,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+replayingreplaying, {"bytes_per_second":3205529.6,"bytes_per_snapshot":64110592.0,"last_snapshot_bytes":64110592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+replayingreplaying, {"bytes_per_second":3378380.8,"bytes_per_snapshot":67567616.0,"last_snapshot_bytes":67567616,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28' 2024-08-25T20:51:47.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:51:47.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:51:47.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:47.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:51:47.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:51:47.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:47.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:51:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary5 2024-08-25T20:51:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:51:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:51:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-08-25T20:51:47.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-08-25T20:51:47.258 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:47.257+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3455380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:51:47.258 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:47.257+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973b80 [3/8e8e6eb4-4c83-4c1e-842f-1191b4695a8c] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:51:48.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:51:48.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-08-25T20:51:48.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:48.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:48.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:51:48.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:48.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:51:48.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary5 2024-08-25T20:51:49.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:49.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary5 2024-08-25T20:51:49.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:51:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:51:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-08-25T20:51:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:51:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:51:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-08-25T20:51:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:51:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:49.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:49.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:49.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:51:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-08-25T20:51:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:51:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:50.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:50.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:50.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:51:50.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:51:50.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 02f58d00-dc8f-41ff-89a9-c8a6ca6f2c72 2024-08-25T20:51:50.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:51:50.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3203891.2,"bytes_per_snapshot":64077824.0,"last_snapshot_bytes":64077824,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"} 2024-08-25T20:51:50.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:51:50.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:51:28' 2024-08-25T20:51:50.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:51:50.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:51:50.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:51:50.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3203891.2,"bytes_per_snapshot":64077824.0,"last_snapshot_bytes":64077824,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"} 2024-08-25T20:51:50.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:51:50.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:50.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:51:50.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:50.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:51:50.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:51:50.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:51:50.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:51:50.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:50.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-08-25 20:51:28' 2024-08-25T20:51:50.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:50.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:50.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:51:50.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:50.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:51:28' 2024-08-25T20:51:50.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:50.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:50.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:51:50.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING541904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+replayingreplaying, {"bytes_per_second":3633971.2,"bytes_per_snapshot":72679424.0,"last_snapshot_bytes":72679424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619037,"remote_snapshot_timestamp":1724619037,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary602f58d00-dc8f-41ff-89a9-c8a6ca6f2c72up+replayingreplaying, {"bytes_per_second":3203891.2,"bytes_per_snapshot":64077824.0,"last_snapshot_bytes":64077824,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619035,"remote_snapshot_timestamp":1724619035,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+replayingreplaying, {"bytes_per_second":3327795.2,"bytes_per_snapshot":66555904.0,"last_snapshot_bytes":66555904,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+replayingreplaying, {"bytes_per_second":3205529.6,"bytes_per_snapshot":64110592.0,"last_snapshot_bytes":64110592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+replayingreplaying, {"bytes_per_second":3378380.8,"bytes_per_snapshot":67567616.0,"last_snapshot_bytes":67567616,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28' 2024-08-25T20:51:50.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:51:50.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:51:50.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:50.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:51:50.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:51:50.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:50.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:51:50.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:50.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:50.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:50.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary6 2024-08-25T20:51:50.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:51:50.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:50.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:51:50.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-08-25T20:51:50.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-08-25T20:51:50.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:50.424+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae36ff6c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:51:50.426 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:50.424+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972500 [3/02f58d00-dc8f-41ff-89a9-c8a6ca6f2c72] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:51:51.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:51:51.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-08-25T20:51:51.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:51.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:51.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:51:51.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:51.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:51:51.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary6 2024-08-25T20:51:51.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:51.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary6 2024-08-25T20:51:51.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:51:51.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:51:51.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-08-25T20:51:51.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:51.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:51.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:51:51.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:51:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-08-25T20:51:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:51:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:51:52.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-08-25T20:51:52.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:52.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:52.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:51:52.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:52.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:52.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:52.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:52.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:52.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:51:52.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:51:52.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: e92661f6-6b6c-46fb-92be-0c323b12c6ab 2024-08-25T20:51:52.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:51:52.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3327795.2,"bytes_per_snapshot":66555904.0,"last_snapshot_bytes":66555904,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"} 2024-08-25T20:51:52.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:51:52.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:51:28' 2024-08-25T20:51:52.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:51:52.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:51:52.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:51:52.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3327795.2,"bytes_per_snapshot":66555904.0,"last_snapshot_bytes":66555904,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"} 2024-08-25T20:51:52.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:51:52.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:52.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:51:52.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:52.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:51:52.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:51:52.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:51:52.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:51:52.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:52.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-08-25 20:51:28' 2024-08-25T20:51:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:51:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:51:28' 2024-08-25T20:51:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:51:53.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING441904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+replayingreplaying, {"bytes_per_second":3633971.2,"bytes_per_snapshot":72679424.0,"last_snapshot_bytes":72679424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619037,"remote_snapshot_timestamp":1724619037,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary7e92661f6-6b6c-46fb-92be-0c323b12c6abup+replayingreplaying, {"bytes_per_second":3327795.2,"bytes_per_snapshot":66555904.0,"last_snapshot_bytes":66555904,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619034,"remote_snapshot_timestamp":1724619034,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+replayingreplaying, {"bytes_per_second":3205529.6,"bytes_per_snapshot":64110592.0,"last_snapshot_bytes":64110592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+replayingreplaying, {"bytes_per_second":3378380.8,"bytes_per_snapshot":67567616.0,"last_snapshot_bytes":67567616,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28' 2024-08-25T20:51:53.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:51:53.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:51:53.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:53.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:51:53.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:51:53.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:53.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:51:53.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:53.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:53.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:53.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary7 2024-08-25T20:51:53.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:51:53.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:53.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:51:53.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-08-25T20:51:53.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-08-25T20:51:53.240 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:53.238+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4b949c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:51:53.240 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:53.238+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973180 [3/e92661f6-6b6c-46fb-92be-0c323b12c6ab] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:51:54.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:51:54.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-08-25T20:51:54.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:54.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:54.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:51:54.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:54.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:51:54.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary7 2024-08-25T20:51:54.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:54.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary7 2024-08-25T20:51:54.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:54.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:51:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-08-25T20:51:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:51:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:55.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:55.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:55.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:51:55.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:51:55.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 0f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416 2024-08-25T20:51:55.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:51:55.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3205529.6,"bytes_per_snapshot":64110592.0,"last_snapshot_bytes":64110592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"} 2024-08-25T20:51:55.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:51:55.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:51:28' 2024-08-25T20:51:55.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:51:55.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:51:55.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:51:55.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3205529.6,"bytes_per_snapshot":64110592.0,"last_snapshot_bytes":64110592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"} 2024-08-25T20:51:55.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:51:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:51:55.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:55.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:51:55.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:51:55.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:51:55.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:51:55.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:55.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-08-25 20:51:28' 2024-08-25T20:51:55.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:55.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:55.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:51:55.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:55.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:51:28' 2024-08-25T20:51:55.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:55.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:55.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:51:56.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING341904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+replayingreplaying, {"bytes_per_second":3633971.2,"bytes_per_snapshot":72679424.0,"last_snapshot_bytes":72679424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619037,"remote_snapshot_timestamp":1724619037,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary80f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416up+replayingreplaying, {"bytes_per_second":3205529.6,"bytes_per_snapshot":64110592.0,"last_snapshot_bytes":64110592,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+replayingreplaying, {"bytes_per_second":3378380.8,"bytes_per_snapshot":67567616.0,"last_snapshot_bytes":67567616,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:28' 2024-08-25T20:51:56.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:51:56.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:51:56.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:28' 2024-08-25T20:51:56.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:51:56.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:51:56.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:56.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:51:56.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:56.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:56.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:56.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary8 2024-08-25T20:51:56.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:51:56.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:56.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:51:56.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-08-25T20:51:56.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-08-25T20:51:56.271 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:56.269+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae387fba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:51:56.271 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:56.269+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972f00 [3/0f64c4f5-5b21-4bfd-8eaa-f33dc9ef2416] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:51:57.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:51:57.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-08-25T20:51:57.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:57.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:57.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:51:57.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:57.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:51:57.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary8 2024-08-25T20:51:57.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:57.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary8 2024-08-25T20:51:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:51:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:51:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-08-25T20:51:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:51:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:51:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-08-25T20:51:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:51:57.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:57.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:57.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:57.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:51:57.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:51:58.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-08-25T20:51:58.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:58.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:58.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:51:58.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:58.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:51:58.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:51:58.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:58.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:58.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:51:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:51:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 60684371-5dff-4e57-aa1c-760325a2a3a3 2024-08-25T20:51:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:51:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":67567616.0,"last_snapshot_bytes":67567616,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"} 2024-08-25T20:51:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:51:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:51:58' 2024-08-25T20:51:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:51:58.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:51:58.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:51:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":67567616.0,"last_snapshot_bytes":67567616,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"} 2024-08-25T20:51:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:51:58.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:58.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:51:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:51:58.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:51:58.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:51:58.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:51:58.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:58' 2024-08-25T20:51:58.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-08-25 20:51:58' 2024-08-25T20:51:58.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:51:58.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:58.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:51:58.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:51:58.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:51:58' 2024-08-25T20:51:58.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:51:58.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:51:58.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:51:59.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING241904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":72679424.0,"last_snapshot_bytes":72679424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619037,"remote_snapshot_timestamp":1724619037,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:58image-primary960684371-5dff-4e57-aa1c-760325a2a3a3up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":67567616.0,"last_snapshot_bytes":67567616,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619039,"remote_snapshot_timestamp":1724619039,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:58' 2024-08-25T20:51:59.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:51:59.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:51:59.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:58' 2024-08-25T20:51:59.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:51:59.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:51:59.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:51:59.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:51:59.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:51:59.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:59.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:51:59.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary9 2024-08-25T20:51:59.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:51:59.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:51:59.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:51:59.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-08-25T20:51:59.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-08-25T20:51:59.257 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:59.255+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adeb98680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:51:59.257 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:51:59.255+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972000 [3/60684371-5dff-4e57-aa1c-760325a2a3a3] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:52:00.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:52:00.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-08-25T20:52:00.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:00.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:00.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:00.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:52:00.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:52:00.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary9 2024-08-25T20:52:00.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:00.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary9 2024-08-25T20:52:00.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:52:00.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:52:00.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:52:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:52:01.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-08-25T20:52:01.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:01.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:01.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:01.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:52:01.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:52:01.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:52:01.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:52:01.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:52:01.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:52:01.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:52:01.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 95f0a085-22a4-4bbc-9daf-f1488ce5a207 2024-08-25T20:52:01.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:52:01.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":72679424.0,"last_snapshot_bytes":72679424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619037,"remote_snapshot_timestamp":1724619037,"replay_state":"idle"} 2024-08-25T20:52:01.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:52:01.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:51:58' 2024-08-25T20:52:01.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:52:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:52:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:52:01.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":72679424.0,"last_snapshot_bytes":72679424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619037,"remote_snapshot_timestamp":1724619037,"replay_state":"idle"} 2024-08-25T20:52:01.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:52:01.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:52:01.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:52:01.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:52:01.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:52:01.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:52:01.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:52:01.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:52:01.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:58' 2024-08-25T20:52:01.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-08-25 20:51:58' 2024-08-25T20:52:01.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:01.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:01.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:01.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:52:01.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:51:58' 2024-08-25T20:52:01.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:52:01.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:52:01.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:52:02.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING141904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1095f0a085-22a4-4bbc-9daf-f1488ce5a207up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":72679424.0,"last_snapshot_bytes":72679424,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619037,"remote_snapshot_timestamp":1724619037,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:51:58' 2024-08-25T20:52:02.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:52:02.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:52:02.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:51:58' 2024-08-25T20:52:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:52:02.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:52:02.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:52:02.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:52:02.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:52:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary10 2024-08-25T20:52:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:52:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-08-25T20:52:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-08-25T20:52:02.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:52:02.222+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4fcb380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:52:02.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:52:02.222+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973900 [3/95f0a085-22a4-4bbc-9daf-f1488ce5a207] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:52:03.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:52:03.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-08-25T20:52:03.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:03.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:03.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:03.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:52:03.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:52:03.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary10 2024-08-25T20:52:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:03.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary10 2024-08-25T20:52:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..10} 2024-08-25T20:52:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS=() 2024-08-25T20:52:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS=() 2024-08-25T20:52:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS=() 2024-08-25T20:52:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS=() 2024-08-25T20:52:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RET=0 2024-08-25T20:52:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:52:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:52:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary1 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:03.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:03.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:03.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:03.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:52:03.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:52:03.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:52:03.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:03.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-08-25T20:52:04.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:52:04.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:04.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:52:04.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:04.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-08-25T20:52:04.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:04.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:04.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:04.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:04.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-08-25T20:52:04.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:52:04.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-08-25T20:52:04.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary1': 2024-08-25T20:52:04.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:52:04.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:52:04.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:52:04.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 38acffcedcd9 2024-08-25T20:52:04.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.38acffcedcd9 2024-08-25T20:52:04.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:52:04.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:52:04.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:52:04.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:52:04.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:52:04 2024 2024-08-25T20:52:04.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:52:04 2024 2024-08-25T20:52:04.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:52:04 2024 2024-08-25T20:52:04.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:52:04.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:52:04.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 2579f173-30f9-4673-a851-61403f4bd16d 2024-08-25T20:52:04.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:52:04.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:52:04.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:52:04.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-08-25T20:52:04.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd0 2024-08-25T20:52:04.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:52:04.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-08-25T20:52:05.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:52:05.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:52:05.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:52:05.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 5226dda0-ada7-446a-a087-36c9e51b784d 2024-08-25T20:52:05.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:52:05.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:52:05.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:05.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:52:05.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:52:05.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:52:05.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:52:05.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:05.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary1 2024-08-25T20:52:05.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-08-25T20:52:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:52:05.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary1 2024-08-25T20:52:05.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:52:05.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-08-25T20:52:05.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:05.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:05.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:05.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:52:05.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary2 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary1 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:52:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-08-25T20:52:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary1 2024-08-25T20:52:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-08-25T20:52:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:52:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:52:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:52:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-08-25T20:52:05.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:52:05.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:05.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:52:05.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:05.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-08-25T20:52:05.445 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:05.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:05.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:05.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:05.446 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-08-25T20:52:05.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:52:05.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-08-25T20:52:05.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:52:05.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2644 2024-08-25T20:52:05.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:05.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-08-25T20:52:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary2': 2024-08-25T20:52:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:52:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:52:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 38bca4e1553b 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.38bca4e1553b 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:52:05 2024 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:52:05 2024 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:52:05 2024 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 4d392b83-7305-4b60-b6ea-84b62c29ac4a 2024-08-25T20:52:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:52:05.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:52:05.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:52:05.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-08-25T20:52:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd1 2024-08-25T20:52:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:52:05.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-08-25T20:52:05.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:52:05.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:52:05.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:52:05.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: fcbafdfc-499a-47b5-8a4e-bb1e59dad4e4 2024-08-25T20:52:05.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:52:05.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:52:05.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:05.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:52:05.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:52:05.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:52:06.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:52:06.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:06.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary2 2024-08-25T20:52:06.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-08-25T20:52:06.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:52:06.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary2 2024-08-25T20:52:06.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:52:06.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-08-25T20:52:06.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary2 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary3 2024-08-25T20:52:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary2 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:06.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:06.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:06.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:06.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:52:06.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:52:06.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:52:06.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:06.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-08-25T20:52:06.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:52:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:52:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-08-25T20:52:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-08-25T20:52:06.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:52:06.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-08-25T20:52:06.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:52:06.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2646 2024-08-25T20:52:06.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:06.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-08-25T20:52:06.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary3': 2024-08-25T20:52:06.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:52:06.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:52:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:52:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 38cb1ac6e03f 2024-08-25T20:52:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.38cb1ac6e03f 2024-08-25T20:52:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:52:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:52:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:52:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:52:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:52:06 2024 2024-08-25T20:52:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:52:06 2024 2024-08-25T20:52:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:52:06 2024 2024-08-25T20:52:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:52:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:52:06.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: b9178aa8-3d6a-41d7-946b-0dc934500683 2024-08-25T20:52:06.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:52:06.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:52:06.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:52:06.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-08-25T20:52:06.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd2 2024-08-25T20:52:06.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:52:06.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-08-25T20:52:06.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:52:06.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:52:06.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:52:06.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 6bb34c7c-e68a-4433-89df-2722c6fb2661 2024-08-25T20:52:06.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:52:06.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:52:06.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:06.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:52:06.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:52:06.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:52:07.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:52:07.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:07.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary3 2024-08-25T20:52:07.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-08-25T20:52:07.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:52:07.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary3 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary3 2024-08-25T20:52:07.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:52:07.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:52:07.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary4 2024-08-25T20:52:07.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-08-25T20:52:07.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary3 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:52:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:52:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:52:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-08-25T20:52:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:52:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:52:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-08-25T20:52:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-08-25T20:52:07.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:52:07.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-08-25T20:52:07.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:52:07.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2648 2024-08-25T20:52:07.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-08-25T20:52:07.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary4': 2024-08-25T20:52:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:52:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:52:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:52:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 38da2359eff9 2024-08-25T20:52:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.38da2359eff9 2024-08-25T20:52:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:52:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:52:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:52:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:52:07.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:52:07 2024 2024-08-25T20:52:07.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:52:07 2024 2024-08-25T20:52:07.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:52:07 2024 2024-08-25T20:52:07.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:52:07.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:52:07.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 890c934e-762e-4439-a4cf-97e4d48d8c7d 2024-08-25T20:52:07.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:52:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:52:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:52:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-08-25T20:52:07.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd3 2024-08-25T20:52:07.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:52:07.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-08-25T20:52:08.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:52:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:52:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:52:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: a01c836c-a199-4844-ad0d-98e2d7689804 2024-08-25T20:52:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:52:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:52:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:52:08.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:52:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:52:08.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:52:08.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:08.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary4 2024-08-25T20:52:08.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-08-25T20:52:08.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:52:08.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary4 2024-08-25T20:52:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:52:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-08-25T20:52:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:52:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:08.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary5 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary4 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary4 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:52:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-08-25T20:52:08.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:52:08.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:52:08.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:08.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-08-25T20:52:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:52:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:52:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-08-25T20:52:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:08.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:08.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-08-25T20:52:08.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:52:08.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-08-25T20:52:08.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:08.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:08.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:08.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:08.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:08.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:52:08.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:52:08.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2650 2024-08-25T20:52:08.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:08.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-08-25T20:52:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary5': 2024-08-25T20:52:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:52:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:52:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 38e977a45f8c 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.38e977a45f8c 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:52:08 2024 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:52:08 2024 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:52:08 2024 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 723168d9-bb4b-4d32-a3d8-99d77786ac4b 2024-08-25T20:52:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:52:08.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:52:08.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:52:08.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-08-25T20:52:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd4 2024-08-25T20:52:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:52:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-08-25T20:52:09.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:52:09.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:52:09.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:52:09.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 00d56ca4-1b18-42c0-93c6-8d429c8fbc1f 2024-08-25T20:52:09.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:52:09.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:52:09.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:09.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:52:09.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:52:09.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:52:09.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:52:09.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:09.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary5 2024-08-25T20:52:09.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-08-25T20:52:09.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:52:09.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary5 2024-08-25T20:52:09.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:52:09.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-08-25T20:52:09.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:09.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:09.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:09.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:09.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:52:09.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary5 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary6 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary5 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:09.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:09.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:52:09.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:52:09.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:52:09.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:09.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-08-25T20:52:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:52:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:52:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-08-25T20:52:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-08-25T20:52:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:52:09.628 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-08-25T20:52:09.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:09.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:52:09.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:09.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:09.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:09.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:52:09.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:52:09.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2653 2024-08-25T20:52:09.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2652 2024-08-25T20:52:09.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:09.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:09.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-08-25T20:52:09.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary6': 2024-08-25T20:52:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:52:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:52:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:52:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 38f9d972c473 2024-08-25T20:52:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.38f9d972c473 2024-08-25T20:52:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:52:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:52:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:52:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:52:09.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:52:09 2024 2024-08-25T20:52:09.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:52:09 2024 2024-08-25T20:52:09.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:52:09 2024 2024-08-25T20:52:09.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:52:09.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:52:09.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: ffdc83df-6651-4668-9561-234d384d3d02 2024-08-25T20:52:09.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:52:10.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:52:10.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:52:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-08-25T20:52:10.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd5 2024-08-25T20:52:10.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:52:10.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-08-25T20:52:10.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:52:10.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:52:10.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:52:10.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 428244bb-dc39-4b8b-8b15-e10c1aa823b5 2024-08-25T20:52:10.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:52:10.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:52:10.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:10.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:52:10.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:52:10.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:52:10.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:52:10.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:10.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary6 2024-08-25T20:52:10.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-08-25T20:52:10.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:52:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary6 2024-08-25T20:52:10.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:52:10.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-08-25T20:52:10.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:10.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:10.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:10.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:10.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:52:10.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:10.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:10.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:10.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:52:10.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:52:10.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:52:10.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:52:10.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary7 2024-08-25T20:52:10.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-08-25T20:52:10.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary6 2024-08-25T20:52:10.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:10.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:10.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary6 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-08-25T20:52:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:52:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:52:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:52:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-08-25T20:52:10.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:52:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:52:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-08-25T20:52:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:10.660 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-08-25T20:52:10.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:10.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:10.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:10.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:10.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:10.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:10.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2656 2024-08-25T20:52:10.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2655 2024-08-25T20:52:10.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:10.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:10.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:52:10.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-08-25T20:52:11.713 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:52:11.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2658 2024-08-25T20:52:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-08-25T20:52:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:11.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:11.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:52:11.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:11.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:11.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:11.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:52:11.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary7': 2024-08-25T20:52:11.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:52:11.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:52:11.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:52:11.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 390a4b96add4 2024-08-25T20:52:11.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.390a4b96add4 2024-08-25T20:52:11.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:52:11.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:52:11.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:52:11.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:52:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:52:10 2024 2024-08-25T20:52:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:52:10 2024 2024-08-25T20:52:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:52:10 2024 2024-08-25T20:52:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:52:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:52:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 393b7e43-14d2-4eab-bbbb-53da05c7fcc6 2024-08-25T20:52:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:52:11.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:52:11.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:52:11.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-08-25T20:52:12.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd6 2024-08-25T20:52:12.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:52:12.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-08-25T20:52:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:52:12.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:52:12.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:52:12.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 7f73f412-c789-49a9-ac82-67fa2ddde44d 2024-08-25T20:52:12.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:52:12.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:52:12.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:12.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:52:12.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:52:12.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:52:12.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:52:12.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:12.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary7 2024-08-25T20:52:12.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-08-25T20:52:12.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:52:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary7 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:52:12.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary7 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary8 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary7 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:12.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-08-25T20:52:12.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:52:12.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:12.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:52:12.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:12.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-08-25T20:52:12.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:12.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:12.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:12.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:12.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-08-25T20:52:12.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:52:12.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-08-25T20:52:12.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:52:12.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2660 2024-08-25T20:52:12.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2659 2024-08-25T20:52:12.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:12.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:12.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-08-25T20:52:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:52:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:12.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:12.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:12.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:52:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary8': 2024-08-25T20:52:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:52:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:52:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:52:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 391fad6a8b01 2024-08-25T20:52:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.391fad6a8b01 2024-08-25T20:52:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:52:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:52:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:52:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:52:12.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:52:12 2024 2024-08-25T20:52:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:52:12 2024 2024-08-25T20:52:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:52:12 2024 2024-08-25T20:52:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:52:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:52:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: eee5a9dc-3b0d-46fe-89e9-dcc0c25140bc 2024-08-25T20:52:12.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:52:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:52:12.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:52:12.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:52:12.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-08-25T20:52:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd7 2024-08-25T20:52:13.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:52:13.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-08-25T20:52:13.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:52:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:52:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:52:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 5795afaa-70c7-43ac-876a-23205a9ab641 2024-08-25T20:52:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:52:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:52:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:52:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:52:13.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:52:13.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:52:13.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:13.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary8 2024-08-25T20:52:13.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-08-25T20:52:13.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:52:13.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary8 2024-08-25T20:52:13.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:52:13.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-08-25T20:52:13.282 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary8 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary9 2024-08-25T20:52:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary8 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:52:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:52:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:52:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-08-25T20:52:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:52:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:52:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-08-25T20:52:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-08-25T20:52:13.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:52:13.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-08-25T20:52:13.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:52:13.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2663 2024-08-25T20:52:13.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2662 2024-08-25T20:52:13.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2664 2024-08-25T20:52:13.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:13.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-08-25T20:52:13.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:13.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:52:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:52:13.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary9': 2024-08-25T20:52:13.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:52:13.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:52:13.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:52:13.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3934e9540700 2024-08-25T20:52:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3934e9540700 2024-08-25T20:52:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:52:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:52:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:52:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:52:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:52:13 2024 2024-08-25T20:52:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:52:13 2024 2024-08-25T20:52:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:52:13 2024 2024-08-25T20:52:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:52:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:52:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 2e55cf60-b17c-4f9f-8466-e6c4722a0dd6 2024-08-25T20:52:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:52:13.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:13.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:52:13.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:13.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:13.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:13.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:52:13.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:52:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:52:13.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-08-25T20:52:14.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd8 2024-08-25T20:52:14.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:52:14.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-08-25T20:52:14.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:52:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:52:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:52:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: e82cb210-6595-4936-a351-c15e20894171 2024-08-25T20:52:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:52:14.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:52:14.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:14.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:52:14.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:52:14.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:52:14.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:52:14.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:14.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary9 2024-08-25T20:52:14.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-08-25T20:52:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:52:14.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary9 2024-08-25T20:52:14.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:52:14.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:52:14.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary9 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary10 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary9 2024-08-25T20:52:14.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-08-25T20:52:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T20:52:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T20:52:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T20:52:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T20:52:14.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-08-25T20:52:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T20:52:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T20:52:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:52:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-08-25T20:52:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T20:52:14.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-08-25T20:52:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:52:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-08-25T20:52:14.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2666 2024-08-25T20:52:14.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T20:52:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:14.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2668 2024-08-25T20:52:14.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2667 2024-08-25T20:52:14.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:14.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:14.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-08-25T20:52:14.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:14.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary10': 2024-08-25T20:52:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T20:52:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T20:52:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T20:52:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 39479be30803 2024-08-25T20:52:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.39479be30803 2024-08-25T20:52:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T20:52:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T20:52:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T20:52:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T20:52:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 20:52:14 2024 2024-08-25T20:52:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 20:52:14 2024 2024-08-25T20:52:14.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 20:52:14 2024 2024-08-25T20:52:14.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T20:52:14.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T20:52:14.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 36517776-fbdb-47e2-b79b-76a572c69ca6 2024-08-25T20:52:14.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T20:52:14.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:52:14.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:52:14.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-08-25T20:52:15.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd9 2024-08-25T20:52:15.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T20:52:15.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-08-25T20:52:15.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T20:52:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T20:52:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T20:52:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 7de1db91-5e30-4362-9d94-28af46435906 2024-08-25T20:52:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T20:52:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T20:52:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T20:52:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T20:52:15.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T20:52:15.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T20:52:15.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T20:52:15.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary10 2024-08-25T20:52:15.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-08-25T20:52:15.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T20:52:15.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary10 2024-08-25T20:52:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T20:52:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-08-25T20:52:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T20:52:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:52:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:52:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 118776 2024-08-25T20:52:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary10 2024-08-25T20:52:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:52:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary10 2024-08-25T20:52:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-08-25T20:52:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:52:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:52:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:52:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:52:15.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2671 2024-08-25T20:52:15.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2670 2024-08-25T20:52:15.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:15.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T20:52:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-08-25T20:52:16.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2673 2024-08-25T20:52:16.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2672 2024-08-25T20:52:16.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:16.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:52:16.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:16.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:16.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:16.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:52:16.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:16.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:16.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:52:16.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:16.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:16.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:16.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:52:16.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:16.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:16.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:16.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:16.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:16.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:16.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:52:17.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:17.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:52:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:52:17.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:17.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:52:17.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:17.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:17.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:17.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:52:17.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:17.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:52:17.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:17.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:17.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:17.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:52:17.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2676 2024-08-25T20:52:17.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2675 2024-08-25T20:52:17.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:17.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2674 2024-08-25T20:52:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:17.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:18.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:52:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:18.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:52:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:18.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:18.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:18.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:18.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2677 2024-08-25T20:52:18.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2679 2024-08-25T20:52:18.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2678 2024-08-25T20:52:19.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:19.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:19.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:19.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:52:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:52:19.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:19.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:52:19.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:19.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:19.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:19.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:52:19.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2681 2024-08-25T20:52:20.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2680 2024-08-25T20:52:20.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:20.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:20.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2683 2024-08-25T20:52:20.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2682 2024-08-25T20:52:20.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:20.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:20.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:20.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:20.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:20.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:20.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:20.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:52:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:52:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:52:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:52:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:52:21.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2686 2024-08-25T20:52:21.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2685 2024-08-25T20:52:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2684 2024-08-25T20:52:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:21.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:21.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:22.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:22.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:52:22.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:22.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:22.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:22.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:52:22.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:22.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:52:22.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:22.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:22.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:22.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:52:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:52:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:52:22.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2687 2024-08-25T20:52:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2689 2024-08-25T20:52:22.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2688 2024-08-25T20:52:22.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:22.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:22.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:23.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:23.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:52:23.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:23.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:23.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:23.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:52:23.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:23.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:23.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:23.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:23.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:23.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2691 2024-08-25T20:52:23.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:23.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2690 2024-08-25T20:52:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:23.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:23.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:52:23.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:23.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:23.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:23.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:52:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:52:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:52:24.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:24.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:52:24.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:24.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:24.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:24.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:52:24.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:24.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:52:24.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:24.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:24.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:24.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:52:24.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2692 2024-08-25T20:52:24.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2693 2024-08-25T20:52:24.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:24.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:24.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:24.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:24.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:24.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:24.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:52:24.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:25.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2694 2024-08-25T20:52:25.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:25.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2695 2024-08-25T20:52:25.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:25.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:25.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:52:25.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:25.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:25.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:25.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:52:25.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:25.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:52:25.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:25.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:25.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:25.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:52:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:52:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:52:26.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:26.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:52:26.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:26.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:26.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:26.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:52:26.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2697 2024-08-25T20:52:26.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:26.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:26.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:26.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:26.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:26.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:26.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2698 2024-08-25T20:52:26.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:27.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2696 2024-08-25T20:52:27.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:27.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:52:27.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:27.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:27.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:27.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:52:27.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:27.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:27.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:52:27.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:27.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:27.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:27.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:52:27.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2699 2024-08-25T20:52:27.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:27.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2700 2024-08-25T20:52:27.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:28.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:28.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:52:28.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:28.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:28.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:28.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:52:28.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:28.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:52:28.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:28.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:28.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:28.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:52:28.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2701 2024-08-25T20:52:28.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:29.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2702 2024-08-25T20:52:29.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:29.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:29.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:52:29.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:29.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:29.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:29.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:52:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:52:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:52:29.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2703 2024-08-25T20:52:29.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2705 2024-08-25T20:52:29.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:29.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:30.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:30.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:30.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:30.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:30.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:30.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:52:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2706 2024-08-25T20:52:30.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:30.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:30.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:52:30.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:30.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:30.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:30.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:52:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:52:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:52:31.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2704 2024-08-25T20:52:31.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:31.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2710 2024-08-25T20:52:31.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2708 2024-08-25T20:52:31.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:31.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:31.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:31.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:31.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:31.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:31.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:31.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:52:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:52:32.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2707 2024-08-25T20:52:32.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:32.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:32.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:52:32.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:32.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:32.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:32.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:52:32.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2709 2024-08-25T20:52:32.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:32.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:32.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:52:32.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:32.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:32.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:32.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:52:33.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2711 2024-08-25T20:52:33.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2712 2024-08-25T20:52:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:52:33.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:33.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:33.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:33.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:52:33.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:33.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:52:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:34.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:52:34.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2713 2024-08-25T20:52:34.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:34.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:34.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:52:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:52:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:52:34.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2714 2024-08-25T20:52:34.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:34.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:35.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2716 2024-08-25T20:52:35.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:35.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:35.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:52:35.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:35.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:35.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:35.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:52:35.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:35.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:52:35.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:35.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:35.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:35.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:52:36.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2715 2024-08-25T20:52:36.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:52:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:36.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:52:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:36.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2719 2024-08-25T20:52:36.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:37.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2718 2024-08-25T20:52:37.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2717 2024-08-25T20:52:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:37.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:37.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:52:37.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:37.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:37.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:37.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:52:37.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:37.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:37.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:52:37.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:37.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:37.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:37.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:52:38.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2720 2024-08-25T20:52:38.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:38.862 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:52:38.861+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf877800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:52:38.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:38.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:52:38.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:38.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:38.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:38.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:52:39.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:39.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:52:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:52:39.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2722 2024-08-25T20:52:39.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2721 2024-08-25T20:52:39.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:39.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:39.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:52:39.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:39.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:39.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:39.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:52:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2724 2024-08-25T20:52:40.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2723 2024-08-25T20:52:40.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:40.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:40.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:40.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:40.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:40.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:52:40.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:40.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:40.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:40.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:52:40.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:40.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:40.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:40.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:52:40.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2726 2024-08-25T20:52:40.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:41.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2725 2024-08-25T20:52:41.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:41.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2727 2024-08-25T20:52:41.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:41.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:52:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:52:41.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2729 2024-08-25T20:52:41.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2728 2024-08-25T20:52:41.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:41.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:42.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2730 2024-08-25T20:52:42.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:42.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:52:42.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:42.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:42.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:42.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:52:42.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:43.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2732 2024-08-25T20:52:43.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:52:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:52:43.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2731 2024-08-25T20:52:43.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:43.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:52:43.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:43.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:43.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:43.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:52:43.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:43.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:43.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:43.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:52:43.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:43.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:43.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:43.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:52:44.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:44.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:52:44.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:44.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:44.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:44.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:52:44.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:44.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:52:44.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:44.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:44.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:44.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:52:44.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2734 2024-08-25T20:52:44.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2733 2024-08-25T20:52:44.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:44.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:44.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:44.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:44.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:44.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:52:44.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:44.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:52:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:52:44.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:45.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:45.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:52:45.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:45.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:45.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:45.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:52:45.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2735 2024-08-25T20:52:45.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:45.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2736 2024-08-25T20:52:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:46.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2737 2024-08-25T20:52:46.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:46.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:52:46.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:46.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:46.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:46.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:52:46.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:46.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:46.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:46.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:46.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:46.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:46.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2739 2024-08-25T20:52:46.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:46.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:47.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2738 2024-08-25T20:52:47.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:47.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:47.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:52:47.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:47.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:47.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:47.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:52:47.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:47.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:52:47.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:47.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:47.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:47.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:52:47.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2741 2024-08-25T20:52:47.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:48.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2740 2024-08-25T20:52:48.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:48.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:48.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:52:48.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:48.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:48.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:48.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:52:49.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:49.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:52:49.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:49.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:49.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:49.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:52:49.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2742 2024-08-25T20:52:49.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:49.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:49.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:52:49.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:49.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:49.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:49.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:52:49.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:49.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:52:49.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:49.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:49.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:49.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:52:50.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2744 2024-08-25T20:52:50.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2743 2024-08-25T20:52:50.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:50.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:50.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:50.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:50.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:50.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:52:50.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:50.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:50.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:50.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:50.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:50.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:50.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:50.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:51.708 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2745 2024-08-25T20:52:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2746 2024-08-25T20:52:51.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:51.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2747 2024-08-25T20:52:51.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2748 2024-08-25T20:52:51.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:51.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2749 2024-08-25T20:52:51.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:51.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2750 2024-08-25T20:52:51.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:51.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:51.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:52:51.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:51.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:51.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:51.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:52:51.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:52.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2751 2024-08-25T20:52:52.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:52.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:52:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:52:53.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2752 2024-08-25T20:52:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:53.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:53.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:52:53.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:53.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:53.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:53.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:52:53.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:53.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:52:53.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:53.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:53.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:53.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:52:54.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:54.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:52:54.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:54.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:54.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:54.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:52:54.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:54.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:52:54.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:54.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:54.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:54.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:52:54.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2753 2024-08-25T20:52:54.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2754 2024-08-25T20:52:54.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:54.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:52:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:52:54.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:54.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:52:54.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:54.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:54.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:54.861 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:52:54.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:54.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:54.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:54.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:54.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:54.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:52:54.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:54.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:54.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:54.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:54.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:54.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:55.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:55.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:52:55.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:55.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:55.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:52:55.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:52:55.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2760 2024-08-25T20:52:55.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2755 2024-08-25T20:52:55.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2756 2024-08-25T20:52:55.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:55.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:55.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:56.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:56.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:52:56.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:56.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:56.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:52:56.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:52:56.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2759 2024-08-25T20:52:56.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2757 2024-08-25T20:52:56.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2758 2024-08-25T20:52:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:56.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:57.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2761 2024-08-25T20:52:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:57.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:57.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:52:57.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:57.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:57.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:52:57.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:52:57.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:57.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:52:57.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:57.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:57.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:52:57.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:52:58.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2762 2024-08-25T20:52:58.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:52:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:52:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:52:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:52:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:58.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:52:58.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:52:58.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:58.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:52:58.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:58.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:58.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:52:58.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:52:59.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2764 2024-08-25T20:52:59.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2763 2024-08-25T20:52:59.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:59.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:52:59.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:59.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:52:59.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:59.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:59.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:52:59.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:52:59.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:59.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:52:59.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:59.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:59.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:52:59.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:52:59.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:52:59.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:52:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:52:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:52:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:52:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2767 2024-08-25T20:53:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:00.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:01.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2765 2024-08-25T20:53:01.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2766 2024-08-25T20:53:01.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:01.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:01.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:01.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:01.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:01.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:01.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:01.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:01.876 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:53:01.875+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae352ec00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:53:02.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2770 2024-08-25T20:53:02.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2768 2024-08-25T20:53:02.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2769 2024-08-25T20:53:02.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:02.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:02.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:02.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2771 2024-08-25T20:53:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:53:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:53:02.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:53:02.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:02.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:53:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:53:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:53:03.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2772 2024-08-25T20:53:03.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:04.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:04.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:53:04.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:04.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:04.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:53:04.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:53:04.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:04.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:53:04.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:04.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:04.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:53:04.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:53:04.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2773 2024-08-25T20:53:04.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2774 2024-08-25T20:53:04.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:04.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:53:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:53:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:53:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:53:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:53:05.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:05.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:53:05.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:05.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:05.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:53:05.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:53:05.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:05.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:05.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2777 2024-08-25T20:53:05.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:05.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:05.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:05.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:05.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2776 2024-08-25T20:53:05.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:05.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:06.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2779 2024-08-25T20:53:06.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2778 2024-08-25T20:53:06.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2781 2024-08-25T20:53:06.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:06.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:06.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:07.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:53:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:53:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:53:07.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:07.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:07.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:07.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:07.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:07.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:08.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2775 2024-08-25T20:53:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2780 2024-08-25T20:53:08.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2782 2024-08-25T20:53:08.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:08.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:08.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:53:08.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:08.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:08.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:53:08.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:53:08.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:08.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:53:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:53:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:53:09.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:09.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:53:09.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:09.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:09.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:53:09.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:53:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:53:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:53:09.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:53:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:09.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:53:09.841 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:53:09.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2783 2024-08-25T20:53:09.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2784 2024-08-25T20:53:09.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:09.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:10.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2788 2024-08-25T20:53:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:11.711 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2786 2024-08-25T20:53:11.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2785 2024-08-25T20:53:11.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:11.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2789 2024-08-25T20:53:11.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:11.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2787 2024-08-25T20:53:11.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:11.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:11.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:11.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:11.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:11.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:11.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:11.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:53:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:53:11.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:53:11.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:11.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:11.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:12.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2790 2024-08-25T20:53:12.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:12.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:53:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:53:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:53:13.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2792 2024-08-25T20:53:13.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2791 2024-08-25T20:53:13.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:13.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:53:13.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:13.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:13.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:53:13.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:13.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:13.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:53:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:53:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:53:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:53:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:53:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:53:14.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:14.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:53:14.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:14.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:14.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:53:14.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:53:14.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:14.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:53:14.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:14.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:14.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:53:14.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:53:14.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2794 2024-08-25T20:53:14.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2793 2024-08-25T20:53:14.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:14.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:14.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2795 2024-08-25T20:53:14.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:15.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2797 2024-08-25T20:53:15.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:15.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:15.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:15.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:15.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:15.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:15.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:15.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2796 2024-08-25T20:53:15.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:16.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2798 2024-08-25T20:53:16.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2799 2024-08-25T20:53:16.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:16.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:53:16.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:16.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:16.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:53:16.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:53:16.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:16.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:16.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:16.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:16.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:16.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:53:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:53:17.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:53:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:17.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:17.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:17.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:17.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:17.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:53:17.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:17.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:17.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:53:17.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:18.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2800 2024-08-25T20:53:18.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2805 2024-08-25T20:53:18.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:18.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:18.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:53:18.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:18.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:18.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:53:18.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:53:18.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:18.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:53:18.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:18.935 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:18.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:53:18.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:53:19.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2802 2024-08-25T20:53:19.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2801 2024-08-25T20:53:19.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:19.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:53:19.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:19.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:19.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:53:19.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:53:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:53:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:53:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:53:19.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:20.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2804 2024-08-25T20:53:20.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2803 2024-08-25T20:53:20.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:20.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:21.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2806 2024-08-25T20:53:21.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2807 2024-08-25T20:53:21.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:21.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:21.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:21.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:21.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:21.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:21.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:21.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:21.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:21.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:53:21.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:21.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:21.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:53:21.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:22.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2808 2024-08-25T20:53:22.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2809 2024-08-25T20:53:22.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:22.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:22.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:22.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:22.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:22.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:22.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:22.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:22.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:53:22.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:22.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:22.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:53:22.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:53:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2810 2024-08-25T20:53:22.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:23.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2811 2024-08-25T20:53:23.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:23.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2812 2024-08-25T20:53:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2813 2024-08-25T20:53:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:53:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:53:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:53:23.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:23.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:23.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:23.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:23.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:23.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:23.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:23.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:24.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2814 2024-08-25T20:53:24.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2815 2024-08-25T20:53:24.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:24.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:53:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:53:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:53:24.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:24.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:53:24.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:24.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:24.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:53:24.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:53:25.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2817 2024-08-25T20:53:25.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:25.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:53:25.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:25.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:25.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:53:25.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:53:25.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2816 2024-08-25T20:53:25.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:53:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:53:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:53:25.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:25.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:25.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:25.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:25.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:25.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:26.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2819 2024-08-25T20:53:26.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:26.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:53:26.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:26.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:26.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:53:26.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:26.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2818 2024-08-25T20:53:26.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:26.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:26.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:26.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:26.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:26.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:26.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:26.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:26.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:26.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:53:26.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:26.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:26.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:53:26.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:53:27.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:27.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:53:27.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:27.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:27.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:53:27.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:53:27.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:27.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:27.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:27.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:27.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:27.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:27.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2820 2024-08-25T20:53:27.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:28.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2821 2024-08-25T20:53:28.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:28.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:53:28.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:28.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:28.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:53:28.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:53:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:53:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:53:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:53:28.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2822 2024-08-25T20:53:28.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2823 2024-08-25T20:53:28.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:28.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:28.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:53:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:53:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:53:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:53:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:53:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:53:30.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2824 2024-08-25T20:53:30.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2825 2024-08-25T20:53:30.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:30.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:30.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:30.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:30.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:30.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:30.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:31.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2827 2024-08-25T20:53:31.740 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2826 2024-08-25T20:53:31.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2829 2024-08-25T20:53:31.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:31.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:31.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2828 2024-08-25T20:53:31.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:31.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:53:31.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:31.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:31.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:53:31.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:31.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:31.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:53:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:53:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:53:32.823 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:53:32.822+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adfa22400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:53:33.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2830 2024-08-25T20:53:33.827 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:53:33.826+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae3416400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:53:33.843 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:53:33.843+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae3a89400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:53:33.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:33.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:33.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:33.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:53:33.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:33.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:33.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:53:33.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:53:34.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:34.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:53:34.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:34.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:34.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:53:34.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:53:34.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:34.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:53:34.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:34.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:34.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:53:34.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:53:34.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2833 2024-08-25T20:53:34.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2832 2024-08-25T20:53:34.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:34.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:34.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:34.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:34.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:53:34.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:53:34.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:34.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:34.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:34.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:53:34.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:53:34.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:34.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:53:34.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:53:34.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2835 2024-08-25T20:53:34.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2834 2024-08-25T20:53:34.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:34.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:35.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2837 2024-08-25T20:53:35.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2836 2024-08-25T20:53:35.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:35.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:36.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2831 2024-08-25T20:53:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2838 2024-08-25T20:53:36.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2839 2024-08-25T20:53:36.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:36.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:36.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:36.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:36.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:36.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:36.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:36.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:37.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2840 2024-08-25T20:53:37.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:37.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:37.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:37.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:53:37.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:37.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:37.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:53:37.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:53:37.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:37.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:53:37.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:37.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:37.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:53:37.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:53:37.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:37.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:37.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:37.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:37.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:37.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:38.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2841 2024-08-25T20:53:38.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2842 2024-08-25T20:53:38.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:38.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:38.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:38.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:53:38.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:38.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:38.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:53:38.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:53:38.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:38.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:53:38.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:38.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:38.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:53:38.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:53:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:53:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:53:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:39.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:39.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:53:39.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:39.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:39.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:53:39.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:53:39.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:39.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:53:39.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:39.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:39.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:53:39.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:53:39.879 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:53:39.878+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae35b2c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:53:40.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2843 2024-08-25T20:53:40.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2844 2024-08-25T20:53:40.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:40.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:40.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:40.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:40.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:40.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:40.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:40.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:41.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2845 2024-08-25T20:53:41.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2846 2024-08-25T20:53:41.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:41.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2847 2024-08-25T20:53:41.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2848 2024-08-25T20:53:41.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2849 2024-08-25T20:53:41.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:41.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:41.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:41.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:41.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2850 2024-08-25T20:53:41.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:41.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:41.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:41.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:41.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:41.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:41.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:41.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:53:41.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:41.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:41.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:53:41.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:53:41.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:42.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2852 2024-08-25T20:53:42.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2851 2024-08-25T20:53:42.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:42.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:43.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:43.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:53:43.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:43.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:43.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:53:43.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:53:43.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:43.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:43.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:43.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:43.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:43.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:44.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:44.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:53:44.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:44.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:44.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:53:44.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:53:44.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:44.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:53:44.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:44.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:44.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:53:44.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:53:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:53:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:53:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:53:44.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:44.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:53:44.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:44.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:44.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:53:44.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:44.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:44.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:53:44.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:44.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:44.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:53:44.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:53:44.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2853 2024-08-25T20:53:44.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2854 2024-08-25T20:53:44.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:44.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:44.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:44.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:44.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:44.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:44.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:44.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:45.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:45.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:45.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:45.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:45.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:45.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:45.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:53:45.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:45.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:45.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:53:45.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:53:45.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2855 2024-08-25T20:53:45.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2860 2024-08-25T20:53:45.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2856 2024-08-25T20:53:45.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:45.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:45.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:46.813 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:53:46.812+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae4ed2400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:53:46.850 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:53:46.822+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adfa48000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:53:46.850 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:53:46.833+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf93cc00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:53:46.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2862 2024-08-25T20:53:46.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2861 2024-08-25T20:53:46.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:46.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:47.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2859 2024-08-25T20:53:47.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2858 2024-08-25T20:53:47.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2857 2024-08-25T20:53:47.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:47.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:47.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:47.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:47.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:47.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:47.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:53:47.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:47.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:47.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:47.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:47.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:53:47.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:53:47.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:47.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:48.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2864 2024-08-25T20:53:48.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:48.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:48.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:53:48.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:48.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:48.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:53:48.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:53:48.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:48.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:48.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:48.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:48.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:48.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:53:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:53:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:53:49.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:49.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:49.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:49.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:49.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:49.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:49.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:49.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:53:49.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:49.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:49.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:53:49.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:53:49.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2865 2024-08-25T20:53:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:50.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:50.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:53:50.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:50.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:50.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:53:50.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:53:50.871 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2863 2024-08-25T20:53:50.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:50.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:53:50.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:50.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:50.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:53:50.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:50.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:50.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:53:50.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:50.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:50.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:53:50.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:53:50.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2868 2024-08-25T20:53:50.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2869 2024-08-25T20:53:50.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:50.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:51.728 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2866 2024-08-25T20:53:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2867 2024-08-25T20:53:51.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:51.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:51.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2870 2024-08-25T20:53:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2872 2024-08-25T20:53:51.791 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2871 2024-08-25T20:53:51.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:51.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:51.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:51.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:51.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:51.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:51.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:51.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:51.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2873 2024-08-25T20:53:52.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:52.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:52.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:52.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:52.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:52.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:52.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:53.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:53.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:53:53.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:53.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:53.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:53:53.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:53:53.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:53.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:53:53.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:53.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:53.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:53:53.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:53:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:53.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:54.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:54.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:53:54.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:54.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:54.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:53:54.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:53:54.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:54.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:53:54.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:54.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:54.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:53:54.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:53:54.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:54.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:53:54.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:54.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:54.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:53:54.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:53:54.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:54.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:53:54.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:54.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:54.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:53:54.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:53:54.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:54.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:53:54.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:54.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:54.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:53:54.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:54.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2874 2024-08-25T20:53:54.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:54.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2875 2024-08-25T20:53:54.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:55.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2876 2024-08-25T20:53:55.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2877 2024-08-25T20:53:55.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:55.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:55.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:55.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:55.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:55.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:55.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:55.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:55.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2879 2024-08-25T20:53:55.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2882 2024-08-25T20:53:55.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2881 2024-08-25T20:53:55.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2878 2024-08-25T20:53:55.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2880 2024-08-25T20:53:55.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:55.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:55.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:55.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:55.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:56.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2883 2024-08-25T20:53:56.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:57.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:57.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:53:57.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:57.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:57.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:53:57.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:53:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:53:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:53:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:53:58.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:58.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:53:58.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:58.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:58.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:53:58.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:53:58.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:58.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:53:58.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:58.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:58.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:53:58.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:53:58.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:58.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:53:58.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:58.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:58.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:53:58.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:53:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:53:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:53:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:53:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:53:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:53:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:53:58.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:58.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:53:58.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:58.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:58.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:53:58.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:53:58.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:58.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:53:58.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:58.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:58.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:53:58.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:53:59.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:53:59.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:53:59.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:53:59.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:53:59.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:53:59.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:53:59.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2884 2024-08-25T20:53:59.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:53:59.977 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:53:59.976+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae4c4e400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:54:00.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2885 2024-08-25T20:54:00.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2889 2024-08-25T20:54:00.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2892 2024-08-25T20:54:00.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2886 2024-08-25T20:54:00.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:00.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2887 2024-08-25T20:54:00.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:00.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2888 2024-08-25T20:54:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:01.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:01.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:01.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2891 2024-08-25T20:54:01.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2890 2024-08-25T20:54:01.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:01.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:01.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2893 2024-08-25T20:54:01.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:02.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:02.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:54:02.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:02.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:02.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:54:02.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:54:03.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:03.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:54:03.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:03.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:03.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:54:03.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:54:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:54:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:54:03.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:54:03.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:03.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:54:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:54:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:54:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:54:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:54:03.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:54:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:54:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:54:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:54:04.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:04.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:54:04.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:04.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:04.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:54:04.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:54:04.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:04.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:54:04.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:04.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:04.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:54:04.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:54:04.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:04.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:54:04.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:04.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:04.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:54:04.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:54:04.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:54:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:54:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:54:04.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2897 2024-08-25T20:54:05.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:06.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2902 2024-08-25T20:54:06.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:06.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2901 2024-08-25T20:54:06.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:07.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2896 2024-08-25T20:54:07.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2900 2024-08-25T20:54:07.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2895 2024-08-25T20:54:07.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:07.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2899 2024-08-25T20:54:07.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:07.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:07.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2894 2024-08-25T20:54:07.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:07.885 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:54:07.884+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae4e6ec00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:54:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:08.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:08.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:54:08.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:08.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:08.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:54:08.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:54:08.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2898 2024-08-25T20:54:08.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:08.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2904 2024-08-25T20:54:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:09.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:09.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:54:09.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:09.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:09.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:54:09.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:54:09.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:09.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:54:09.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:09.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:09.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:54:09.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:54:09.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2903 2024-08-25T20:54:09.764 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:10.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2906 2024-08-25T20:54:10.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2905 2024-08-25T20:54:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:54:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:54:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:54:10.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:10.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:10.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:10.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:54:10.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:10.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:10.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:54:10.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:54:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:54:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:54:10.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:54:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:54:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:10.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:54:10.847 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:54:10.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:10.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:54:10.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:10.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:10.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:54:10.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:54:11.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:11.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:54:11.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:11.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:11.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:54:11.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:54:11.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2909 2024-08-25T20:54:11.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2907 2024-08-25T20:54:11.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2908 2024-08-25T20:54:11.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:11.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:54:11.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:11.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:11.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:54:11.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:54:11.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:11.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:11.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:12.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:12.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:54:12.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:12.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:12.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:54:12.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:54:12.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2911 2024-08-25T20:54:12.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2913 2024-08-25T20:54:12.846 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2912 2024-08-25T20:54:12.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:12.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:12.886 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:13.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:13.799 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:54:13.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:13.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:13.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:54:13.800 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:54:13.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:13.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:54:13.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:13.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:13.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:54:13.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:54:14.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2910 2024-08-25T20:54:14.790 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:54:14.789+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae4ed2400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:54:14.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:54:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:14.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:54:14.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:54:14.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:14.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:54:14.815 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:14.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:14.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:54:14.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:54:14.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:14.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:54:14.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:14.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:14.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:54:14.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:54:15.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2914 2024-08-25T20:54:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:54:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:54:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:54:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:54:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:54:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:54:15.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:15.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:54:15.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:15.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:15.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:54:15.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:54:16.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2915 2024-08-25T20:54:16.773 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2916 2024-08-25T20:54:16.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:16.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:16.854 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:54:16.853+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae35b3c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:54:16.873 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:54:16.872+0000 7f8837000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adfb16c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:54:16.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2922 2024-08-25T20:54:16.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2920 2024-08-25T20:54:16.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:16.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:17.769 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2918 2024-08-25T20:54:17.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2917 2024-08-25T20:54:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2919 2024-08-25T20:54:17.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:17.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:54:17.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:17.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:17.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:54:17.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:54:17.801 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:54:17.800+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adea66800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:54:17.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:17.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:17.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:18.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2921 2024-08-25T20:54:18.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:18.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:54:18.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:18.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:18.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:54:18.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:54:18.795 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:18.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2923 2024-08-25T20:54:18.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2924 2024-08-25T20:54:19.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:19.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:54:19.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:19.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:19.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:54:19.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:54:19.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:19.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:19.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:54:19.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:19.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:19.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:54:19.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:54:19.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:19.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:54:19.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:19.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:19.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:54:19.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:54:19.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:19.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:54:19.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:19.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:19.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:54:19.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:54:20.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:20.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:54:20.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:20.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:20.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:54:20.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:54:20.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2925 2024-08-25T20:54:20.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2926 2024-08-25T20:54:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:54:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:54:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:54:20.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2927 2024-08-25T20:54:20.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:20.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:54:20.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:20.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:20.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:54:20.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:54:20.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:20.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:20.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:21.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2928 2024-08-25T20:54:21.767 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:21.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2929 2024-08-25T20:54:21.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2931 2024-08-25T20:54:21.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:21.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:54:21.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:21.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:21.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:54:21.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:54:21.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2930 2024-08-25T20:54:21.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:21.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:21.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:21.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:21.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:54:21.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:21.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:21.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:54:21.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:54:22.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2932 2024-08-25T20:54:22.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:22.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:54:22.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:22.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:22.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:54:22.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:54:22.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:23.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2934 2024-08-25T20:54:23.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:23.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:54:23.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:23.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:23.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:54:23.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:54:23.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:23.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:54:23.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:23.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:23.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:23.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:54:23.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:54:23.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:54:23.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:23.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:23.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:54:23.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:54:23.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:24.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:24.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:54:24.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:24.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:24.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:54:24.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:54:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2933 2024-08-25T20:54:24.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:24.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T20:54:24.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:24.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:24.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:54:24.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T20:54:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:54:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:54:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:54:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:54:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:24.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:54:24.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:54:24.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:25.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2936 2024-08-25T20:54:25.826 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:25.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T20:54:25.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:25.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:25.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:54:25.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T20:54:25.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:25.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2940 2024-08-25T20:54:25.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2939 2024-08-25T20:54:25.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:25.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:26.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2935 2024-08-25T20:54:26.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2938 2024-08-25T20:54:26.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2937 2024-08-25T20:54:26.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:26.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:26.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:26.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:54:26.787+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555add606800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T20:54:26.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2942 2024-08-25T20:54:26.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:26.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T20:54:26.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:26.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:26.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:54:26.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T20:54:26.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:27.812 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2941 2024-08-25T20:54:27.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:27.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:27.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T20:54:27.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:27.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:27.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:54:27.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T20:54:27.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2943 2024-08-25T20:54:27.857 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:28.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2944 2024-08-25T20:54:28.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:28.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:28.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T20:54:28.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:28.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:28.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:54:28.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T20:54:28.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:28.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:54:28.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:28.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:28.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:54:28.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:54:28.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:28.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T20:54:28.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:28.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:28.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:54:28.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T20:54:29.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:29.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:54:29.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:29.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:29.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:54:29.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:54:29.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:29.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T20:54:29.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:29.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:29.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:54:29.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T20:54:29.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:29.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T20:54:29.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:29.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:29.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:54:29.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T20:54:30.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:54:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 118908 2024-08-25T20:54:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:54:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119040 2024-08-25T20:54:30.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2946 2024-08-25T20:54:30.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:30.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2947 2024-08-25T20:54:30.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2945 2024-08-25T20:54:30.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:30.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2949 2024-08-25T20:54:30.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2950 2024-08-25T20:54:30.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:30.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:30.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:31.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2948 2024-08-25T20:54:31.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:31.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:54:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119172 2024-08-25T20:54:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:54:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119324 2024-08-25T20:54:33.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:33.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T20:54:33.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:33.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:33.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:54:33.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T20:54:33.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:54:33.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119477 2024-08-25T20:54:33.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:54:33.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119651 2024-08-25T20:54:33.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:54:33.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119824 2024-08-25T20:54:33.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T20:54:33.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T20:54:33.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:54:33.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:54:33.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:54:33.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T20:54:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:54:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119998 2024-08-25T20:54:36.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2952 2024-08-25T20:54:36.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:37.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2951 2024-08-25T20:54:37.757 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T20:54:39.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T20:54:39.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 120152 2024-08-25T20:54:40.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:54:40.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:54:40.761 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 118777 2024-08-25T20:57:05.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:57:05.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:57:05.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:57:05.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 118909 2024-08-25T20:57:06.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:57:06.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:57:06.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:57:06.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119041 2024-08-25T20:57:07.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:57:07.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:57:07.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:57:07.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119173 2024-08-25T20:57:08.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:57:08.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:57:08.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:57:08.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119325 2024-08-25T20:57:09.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:57:09.631 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:57:09.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:57:09.632 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119478 2024-08-25T20:57:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:57:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:57:10.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:57:10.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119652 2024-08-25T20:57:12.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:57:12.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:57:12.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:57:12.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119825 2024-08-25T20:57:13.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:57:13.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:57:13.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:57:13.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 119999 2024-08-25T20:57:14.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:57:14.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:57:14.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T20:57:14.607 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 120153 2024-08-25T20:57:15.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T20:57:15.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T20:57:15.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:57:15.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:57:15.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:57:15.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:57:15.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 1 2024-08-25T20:57:15.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd0 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary1 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary1 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary1 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 2 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd1 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary2 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary2 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary2 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:57:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:57:15.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 3 2024-08-25T20:57:15.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd2 2024-08-25T20:57:15.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary3 2024-08-25T20:57:15.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary3 2024-08-25T20:57:15.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:57:15.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary3 2024-08-25T20:57:15.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:57:15.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:57:15.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 4 2024-08-25T20:57:15.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd3 2024-08-25T20:57:15.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary4 2024-08-25T20:57:15.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary4 2024-08-25T20:57:15.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:57:15.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary4 2024-08-25T20:57:15.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:57:15.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:57:15.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 5 2024-08-25T20:57:15.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd4 2024-08-25T20:57:15.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary5 2024-08-25T20:57:15.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary5 2024-08-25T20:57:15.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:57:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary5 2024-08-25T20:57:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:57:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:57:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 6 2024-08-25T20:57:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd5 2024-08-25T20:57:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary6 2024-08-25T20:57:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary6 2024-08-25T20:57:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:57:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary6 2024-08-25T20:57:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:57:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:57:15.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 7 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd6 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary7 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary7 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary7 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 8 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd7 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary8 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary8 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary8 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:57:15.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 9 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd8 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary9 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary9 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary9 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 10 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 126083 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd9 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary10 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary10 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T20:57:15.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary10 2024-08-25T20:57:16.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:57:16.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:57:16.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:57:16.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:57:16.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:57:16.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:57:16.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:57:16.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:57:16.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:57:16.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:57:16.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:57:16.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:57:16.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:57:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:57:16.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:57:16.756 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:57:16.776 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:57:16.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:57:16.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:57:16.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:58:09.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=7f8c4269250db5aae669c257980987ae 2024-08-25T20:58:09.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-08-25T20:58:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary7 2024-08-25T20:58:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:58:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-08-25T20:58:11.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=302e15489bf8458ff51f56547b8de909 2024-08-25T20:58:11.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-08-25T20:58:11.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=5f115928c88fb408a534ff321101a8af 2024-08-25T20:58:11.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-08-25T20:58:11.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=b6c2d26eff95d8a35a53738e983443ce 2024-08-25T20:58:11.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-08-25T20:58:11.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary4 2024-08-25T20:58:11.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:11.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:11.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:58:11.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-08-25T20:58:11.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary2 2024-08-25T20:58:11.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:11.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:11.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:58:11.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-08-25T20:58:11.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary3 2024-08-25T20:58:11.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:11.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:11.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:58:11.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-08-25T20:58:11.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=b718c4178e9637ca8c6af931ebd9bb3a 2024-08-25T20:58:11.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-08-25T20:58:12.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary9 2024-08-25T20:58:12.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:12.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:12.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:58:12.288 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-08-25T20:58:12.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=906c41dcb5d9c32c1a95849a7192894d 2024-08-25T20:58:12.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-08-25T20:58:12.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=9b418e0b72721f06a92c35baa9ecf92e 2024-08-25T20:58:12.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-08-25T20:58:12.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary8 2024-08-25T20:58:12.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:12.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:12.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:58:12.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-08-25T20:58:12.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=ec58ffca7d6fc410cc867fba0500a0a6 2024-08-25T20:58:12.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-08-25T20:58:12.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=15088d778f7b8b64426a39e297aeebb7 2024-08-25T20:58:12.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-08-25T20:58:12.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=5df306e05f6677f8ecf4139dff5ecebd 2024-08-25T20:58:12.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-08-25T20:58:12.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary6 2024-08-25T20:58:12.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:12.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:12.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:58:12.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-08-25T20:58:12.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary1 2024-08-25T20:58:12.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:12.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:12.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:58:12.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-08-25T20:58:12.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary5 2024-08-25T20:58:12.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:12.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:12.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:58:12.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-08-25T20:58:12.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary10 2024-08-25T20:58:12.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:12.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:12.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:58:12.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-08-25T20:58:13.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:58:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:58:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-08-25T20:58:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:58:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:58:13.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:58:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-08-25T20:58:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:58:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-08-25T20:58:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:58:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:13.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:13.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-08-25T20:58:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:58:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:13.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:58:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:58:13.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:58:14.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-08-25T20:58:14.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:14.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:14.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:58:14.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:14.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:14.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:14.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:14.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:14.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-08-25T20:58:14.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:14.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:14.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:58:14.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:14.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:14.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:14.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:14.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:14.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-08-25T20:58:14.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:14.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:14.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:58:14.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:14.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:14.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:14.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:14.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:14.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:58:14.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:14.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:14.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:58:14.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:14.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:14.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:14.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:14.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:58:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:58:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:58:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:14.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:14.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:58:14.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:58:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:58:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:14.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:58:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:58:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:14.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:58:14.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:14.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:14.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:14.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:14.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:14.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:58:14.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:58:14.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9178aa8-3d6a-41d7-946b-0dc934500683 2024-08-25T20:58:14.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:14.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:14.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:14.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:14.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:14.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:14.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:14.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:58:14.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:58:14.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 2e55cf60-b17c-4f9f-8466-e6c4722a0dd6 2024-08-25T20:58:14.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:14.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:14.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:14.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:14.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:14.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:14.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:14.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:58:14.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:58:14.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 2579f173-30f9-4673-a851-61403f4bd16d 2024-08-25T20:58:14.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:14.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:14.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:14.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:14.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:58:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:58:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 36517776-fbdb-47e2-b79b-76a572c69ca6 2024-08-25T20:58:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:14.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:14.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:14.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:58:14.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:58:15.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:58:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:58:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:58:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:15.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:15.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:58:15.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-08-25T20:58:15.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:15.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:15.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:58:15.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:15.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:15.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:15.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:15.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:15.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:15.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:15.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:15.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:15.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:15.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:58:15.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:58:15.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:15.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:15.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:58:15.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:15.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:15.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:15.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:15.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:15.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:58:15.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:58:15.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 393b7e43-14d2-4eab-bbbb-53da05c7fcc6 2024-08-25T20:58:15.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:15.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:15.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:15.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:15.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:15.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:15.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:15.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:58:15.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:58:15.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d392b83-7305-4b60-b6ea-84b62c29ac4a 2024-08-25T20:58:15.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:15.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:15.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:15.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:58:15.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 723168d9-bb4b-4d32-a3d8-99d77786ac4b 2024-08-25T20:58:15.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:15.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:15.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:15.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:15.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:15.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:15.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:15.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:15.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:15.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:58:15.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:15.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:15.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:58:16.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:58:16.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:16.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:16.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:58:16.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:16.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:16.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:16.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:16.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:16.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:58:16.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:58:16.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:16.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:16.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:58:16.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:16.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:16.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:16.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:16.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:16.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:58:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:58:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 890c934e-762e-4439-a4cf-97e4d48d8c7d 2024-08-25T20:58:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:16.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:16.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:16.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:58:16.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:58:16.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ffdc83df-6651-4668-9561-234d384d3d02 2024-08-25T20:58:16.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:16.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:16.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:16.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:16.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:16.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:16.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:16.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:58:16.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:58:16.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:16.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:16.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:58:16.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:16.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:16.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:16.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:16.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:16.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:58:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:58:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9178aa8-3d6a-41d7-946b-0dc934500683 2024-08-25T20:58:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:16.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:16.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:16.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:58:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:58:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:58:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:16.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:16.440 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:58:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:58:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:58:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:16.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:16.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:58:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:58:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:58:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:16.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:16.480 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:58:16.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:58:16.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 2e55cf60-b17c-4f9f-8466-e6c4722a0dd6 2024-08-25T20:58:16.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:16.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:16.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:16.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:16.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:16.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:16.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:16.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:58:16.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:58:16.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 2579f173-30f9-4673-a851-61403f4bd16d 2024-08-25T20:58:16.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:16.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:16.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:16.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:16.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:16.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:16.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:16.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:58:16.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:58:16.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 36517776-fbdb-47e2-b79b-76a572c69ca6 2024-08-25T20:58:16.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:16.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:16.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:16.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:16.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:16.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:16.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:16.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:58:16.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T20:58:16.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-08-25T20:58:16.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:16.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:16.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:58:16.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:16.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:16.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:16.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:16.786 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:17.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:58:17.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:17.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:17.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:58:17.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:17.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:17.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:17.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:17.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:17.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:58:17.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:58:17.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:17.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:17.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:58:17.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:17.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:17.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:17.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:17.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:17.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:58:17.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:58:17.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:17.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:17.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:58:17.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:17.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:17.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:17.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:17.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:17.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:58:17.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:58:17.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 393b7e43-14d2-4eab-bbbb-53da05c7fcc6 2024-08-25T20:58:17.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:17.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:17.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:17.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:17.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:17.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:17.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:17.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:58:17.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:58:17.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 723168d9-bb4b-4d32-a3d8-99d77786ac4b 2024-08-25T20:58:17.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:17.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:17.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:17.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:17.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:17.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:58:17.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d392b83-7305-4b60-b6ea-84b62c29ac4a 2024-08-25T20:58:17.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:17.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:17.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:17.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:17.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:17.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:17.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:17.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:58:17.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:17.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:17.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:58:17.788 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:58:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:58:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:58:17.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:58:17.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: eee5a9dc-3b0d-46fe-89e9-dcc0c25140bc 2024-08-25T20:58:17.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:17.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:17.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:17.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:17.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:17.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:17.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:17.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T20:58:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:58:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:58:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:18.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:18.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:58:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:58:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:58:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:58:18.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:58:18.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 890c934e-762e-4439-a4cf-97e4d48d8c7d 2024-08-25T20:58:18.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:18.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:18.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:18.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:18.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:18.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:58:18.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ffdc83df-6651-4668-9561-234d384d3d02 2024-08-25T20:58:18.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:18.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:18.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:18.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:18.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:18.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:18.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:18.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:58:18.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:18.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:18.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:58:19.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:58:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:58:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:58:19.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:58:19.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: eee5a9dc-3b0d-46fe-89e9-dcc0c25140bc 2024-08-25T20:58:19.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:19.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:19.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:19.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:19.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:19.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:19.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:19.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T20:58:20.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:58:20.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:20.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:20.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:58:20.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:20.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:20.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:20.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:20.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:20.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:58:20.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:58:20.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9178aa8-3d6a-41d7-946b-0dc934500683 2024-08-25T20:58:20.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:20.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:20.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:20.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:20.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:20.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:20.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:20.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:58:20.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:58:20.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:20.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:20.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:58:20.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:20.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:20.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:20.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:20.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:20.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:58:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:58:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:58:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:58:20.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:58:20.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:20.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:20.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:58:20.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:20.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:20.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:20.559 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:20.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:20.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:58:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:58:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 2e55cf60-b17c-4f9f-8466-e6c4722a0dd6 2024-08-25T20:58:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:20.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:20.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:20.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:58:20.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:58:20.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 2579f173-30f9-4673-a851-61403f4bd16d 2024-08-25T20:58:20.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:20.617 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:20.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:20.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:58:20.633 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:58:20.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 36517776-fbdb-47e2-b79b-76a572c69ca6 2024-08-25T20:58:20.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:20.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:20.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:20.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:20.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:20.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:20.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:20.637 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:58:21.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:58:21.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:21.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:21.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:58:21.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:21.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:21.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:21.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:21.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:21.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:58:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:58:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:58:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:58:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:58:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:58:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:21.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:58:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:58:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 393b7e43-14d2-4eab-bbbb-53da05c7fcc6 2024-08-25T20:58:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:58:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 723168d9-bb4b-4d32-a3d8-99d77786ac4b 2024-08-25T20:58:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:21.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:21.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:21.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:58:21.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:58:21.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d392b83-7305-4b60-b6ea-84b62c29ac4a 2024-08-25T20:58:21.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:21.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:21.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:21.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:21.334 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:21.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:21.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:21.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:58:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:58:22.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:58:22.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:22.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:22.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:58:22.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:58:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:58:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:22.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:22.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:58:22.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:22.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:58:22.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:58:22.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ffdc83df-6651-4668-9561-234d384d3d02 2024-08-25T20:58:22.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:22.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:22.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:22.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:22.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:22.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:22.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:22.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:58:22.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:58:22.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 890c934e-762e-4439-a4cf-97e4d48d8c7d 2024-08-25T20:58:22.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:22.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:22.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:22.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:22.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:22.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:22.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:22.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:58:23.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:58:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:58:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:58:24.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:58:24.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: eee5a9dc-3b0d-46fe-89e9-dcc0c25140bc 2024-08-25T20:58:24.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T20:58:24.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T20:58:24.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:24.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:57:58' 2024-08-25T20:58:24.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:24.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T20:58:24.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:24.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T20:58:28.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T20:58:28.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:28.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:28.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:58:28.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:28.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:28.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:28.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:28.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:28.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:58:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:58:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9178aa8-3d6a-41d7-946b-0dc934500683 2024-08-25T20:58:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:28.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:28.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:28.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:28.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:28.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:28.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:28.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:28.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:28.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:28.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:28.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:28.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:28.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:28.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:58:28.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:28.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:28.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:28.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:28.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:28.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T20:58:28.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:28.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:28.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:58:28.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:28.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:28.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:28.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:28.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:28.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:58:28.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T20:58:28.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:28.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:28.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:58:28.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:28.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:28.621 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:28.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:28.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:28.622 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:58:28.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T20:58:28.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:28.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:28.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:58:28.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:28.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:28.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:28.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:28.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:28.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:58:28.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:58:28.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 2e55cf60-b17c-4f9f-8466-e6c4722a0dd6 2024-08-25T20:58:28.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:28.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:28.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:28.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:28.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:28.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:28.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:28.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:28.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:28.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:28.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:28.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:28.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:28.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:28.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:58:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:28.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:28.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:28.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:28.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:28.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:58:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 2579f173-30f9-4673-a851-61403f4bd16d 2024-08-25T20:58:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:28.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:28.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:28.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:28.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:28.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:58:28.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:28.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:28.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:28.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:28.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:28.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:28.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:28.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:58:28.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 36517776-fbdb-47e2-b79b-76a572c69ca6 2024-08-25T20:58:28.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:28.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:28.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:28.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:28.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:28.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:28.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:28.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:28.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:28.729 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:28.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:28.731 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:28.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:28.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:28.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:28.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:28.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:28.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:28.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:28.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:58:28.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:28.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:28.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:28.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:28.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:28.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:28.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:28.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:28.737 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:28.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:28.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:28.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:28.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:28.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:28.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:58:28.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:28.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:28.742 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:28.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:28.743 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:28.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:28.772 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:58:28.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:28.777 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:28.778 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:28.780 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:28.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:28.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:28.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-08-25T20:58:28.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:28.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:28.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:58:28.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:28.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:28.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:28.781 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:28.782 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:28.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:28.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:28.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:58:28.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:28.822 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:58:28.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:28.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:28.828 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:28.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:28.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:28.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:28.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-08-25T20:58:28.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:28.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:28.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:58:28.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:28.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:28.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:28.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:28.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:28.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:28.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:28.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:58:28.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:28.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:58:28.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:28.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:28.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:28.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:28.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:28.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:28.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-08-25T20:58:28.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:28.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:28.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:58:28.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:28.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:28.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:28.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:28.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:28.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:28.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:28.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:58:28.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:28.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:58:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:28.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-08-25T20:58:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:58:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:28.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:28.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:29.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T20:58:29.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:29.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:58:29.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:29.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:29.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:29.336 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:58:29.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T20:58:29.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:29.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:58:29.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:29.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:29.338 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:58:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T20:58:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:58:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:58:29.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:58:29.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 393b7e43-14d2-4eab-bbbb-53da05c7fcc6 2024-08-25T20:58:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:29.411 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:29.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:58:29.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 723168d9-bb4b-4d32-a3d8-99d77786ac4b 2024-08-25T20:58:29.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:29.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:29.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:29.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:29.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:29.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:58:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d392b83-7305-4b60-b6ea-84b62c29ac4a 2024-08-25T20:58:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:29.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:29.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:29.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:29.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:29.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:29.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:29.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:29.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:29.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:29.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:29.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:29.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:29.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:29.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:29.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:29.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:29.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:29.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:29.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:29.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:29.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:29.424 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:29.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:29.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:58:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:29.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:29.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:58:29.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:29.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:58:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:29.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:29.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:29.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:58:29.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:29.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:29.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:58:29.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:29.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:58:29.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:29.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:29.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:58:29.588 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:29.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:58:29.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:29.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:29.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:29.592 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:29.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:58:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-08-25T20:58:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:58:29.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:29.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:29.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:29.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:29.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:29.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:29.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-08-25T20:58:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:58:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:29.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:29.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:29.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:29.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:29.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:29.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:29.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-08-25T20:58:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:58:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:29.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-08-25T20:58:29.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:29.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.784 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:58:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-08-25T20:58:29.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-08-25T20:58:29.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:29.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:58:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-08-25T20:58:29.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:58:29.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9178aa8-3d6a-41d7-946b-0dc934500683 2024-08-25T20:58:29.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:29.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:29.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:29.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:29.875 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:29.876 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:29.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:29.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:29.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:29.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:29.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:29.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-08-25T20:58:29.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:29.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:58:29.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:29.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:29.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:29.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:29.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:29.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-08-25T20:58:29.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:29.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:29.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:29.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:29.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:29.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:29.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:58:29.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:29.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:29.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:29.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-08-25T20:58:29.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:29.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:58:29.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:29.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:29.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:29.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-08-25T20:58:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:58:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 2e55cf60-b17c-4f9f-8466-e6c4722a0dd6 2024-08-25T20:58:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:29.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:29.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:29.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:29.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:29.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:29.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:29.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:29.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:29.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:29.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:29.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:29.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:29.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:29.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:29.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:29.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:29.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:29.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:58:29.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:29.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:29.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:29.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:58:29.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 2579f173-30f9-4673-a851-61403f4bd16d 2024-08-25T20:58:29.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:29.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:29.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:29.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:29.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:29.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:29.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:29.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:29.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:29.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:29.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:58:29.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 36517776-fbdb-47e2-b79b-76a572c69ca6 2024-08-25T20:58:29.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:29.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:29.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:29.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:29.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:29.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:29.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:29.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:29.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:29.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:29.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:29.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:29.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:58:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:29.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:30.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:30.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:30.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:30.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:30.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:30.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:30.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:58:30.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:30.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:30.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:30.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:30.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:30.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:58:30.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:58:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:30.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary3 2024-08-25T20:58:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:58:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:58:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-08-25T20:58:30.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:30.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:30.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:58:30.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:58:30.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:30.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:30.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary9 2024-08-25T20:58:30.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:58:30.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:58:30.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-08-25T20:58:30.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:30.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:30.191 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:58:30.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:58:30.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:30.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:30.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:58:30.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:30.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:30.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:58:30.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary10 2024-08-25T20:58:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:58:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:58:30.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-08-25T20:58:30.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:30.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:30.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary1 2024-08-25T20:58:30.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:58:30.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:58:30.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-08-25T20:58:30.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T20:58:30.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:30.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:58:30.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:30.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:30.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:30.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:30.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:58:30.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T20:58:30.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:30.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:58:30.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:30.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:30.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:58:30.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:58:30.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ffdc83df-6651-4668-9561-234d384d3d02 2024-08-25T20:58:30.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:30.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:30.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:30.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:30.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:30.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:58:30.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 890c934e-762e-4439-a4cf-97e4d48d8c7d 2024-08-25T20:58:30.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:30.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:30.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:30.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:30.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:30.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:30.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:30.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:30.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:30.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:30.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:30.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:30.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:30.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:30.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:30.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.395 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:30.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:30.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:30.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:30.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:30.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:30.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:58:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:30.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:30.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:30.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:58:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:30.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:30.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:30.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:58:30.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:30.532 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:30.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:58:30.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:58:30.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.537 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:58:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:30.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:30.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:30.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-08-25T20:58:30.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:58:30.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.544 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:30.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:30.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:30.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:30.545 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-08-25T20:58:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:58:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:30.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-08-25T20:58:30.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:58:30.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:30.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:30.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:30.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:30.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-08-25T20:58:30.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-08-25T20:58:30.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:58:30.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.599 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:30.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:30.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:30.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:30.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-08-25T20:58:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-08-25T20:58:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:58:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-08-25T20:58:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:58:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 723168d9-bb4b-4d32-a3d8-99d77786ac4b 2024-08-25T20:58:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:30.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:30.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:30.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:30.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:58:30.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 393b7e43-14d2-4eab-bbbb-53da05c7fcc6 2024-08-25T20:58:30.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:30.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:30.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:30.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:30.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:30.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:58:30.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d392b83-7305-4b60-b6ea-84b62c29ac4a 2024-08-25T20:58:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:30.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:30.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:30.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:30.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:30.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:30.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:30.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:30.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:30.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:30.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:30.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:30.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:30.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:30.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:30.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:30.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:30.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:30.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:58:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:30.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:30.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:58:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:30.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:30.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:30.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:30.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:30.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:58:30.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:30.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:30.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:30.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:30.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:30.866 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:58:30.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:58:30.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:58:30.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:30.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:30.879 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:58:30.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:58:30.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:30.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:30.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:58:30.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:58:30.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:30.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:30.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:58:30.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:58:30.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:58:30.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-08-25T20:58:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:58:30.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:30.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:58:30.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:58:30.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:30.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-08-25T20:58:30.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:58:30.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary5 2024-08-25T20:58:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:58:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:58:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-08-25T20:58:30.902 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary7 2024-08-25T20:58:30.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:58:30.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:58:30.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-08-25T20:58:30.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:58:30.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:58:30.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:30.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:58:30.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:58:30.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-08-25T20:58:30.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:30.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:30.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-08-25T20:58:30.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:30.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:30.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary2 2024-08-25T20:58:30.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:30.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:30.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:58:30.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:58:30.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-08-25T20:58:31.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd3 2024-08-25T20:58:31.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd2 2024-08-25T20:58:31.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:58:31.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T20:58:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T20:58:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:58:31.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd0 2024-08-25T20:58:31.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd1 2024-08-25T20:58:31.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:58:31.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T20:58:31.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:58:31.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T20:58:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-08-25T20:58:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:58:31.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:31.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:31.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:31.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:31.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-08-25T20:58:31.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:31.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-08-25T20:58:31.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:31.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:31.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:58:31.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:31.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:31.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:31.548 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:31.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:31.549 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-08-25T20:58:31.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:58:31.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 890c934e-762e-4439-a4cf-97e4d48d8c7d 2024-08-25T20:58:31.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:31.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:31.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:31.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:31.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:31.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:58:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ffdc83df-6651-4668-9561-234d384d3d02 2024-08-25T20:58:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:31.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:31.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:31.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:31.671 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:31.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:31.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:31.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:31.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:31.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:31.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:31.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:31.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:31.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:31.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:31.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:31.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:31.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:31.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:31.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:31.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:31.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:31.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:31.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:31.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:58:31.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:31.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:31.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:31.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:31.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:58:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:31.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:31.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:31.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:58:31.880 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:58:31.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:58:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:31.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:58:31.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:58:31.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:58:31.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:31.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-08-25T20:58:31.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:58:31.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:58:31.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:58:31.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-08-25T20:58:31.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:58:31.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:58:31.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-08-25T20:58:31.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:31.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:31.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:31.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:31.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:31.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:31.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary6 2024-08-25T20:58:31.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:31.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:31.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:58:31.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:58:31.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-08-25T20:58:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:31.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:58:31.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:31.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:58:31.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:31.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:31.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:31.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:31.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:31.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:31.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary4 2024-08-25T20:58:31.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:31.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:31.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:58:31.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:58:31.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-08-25T20:58:32.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T20:58:32.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:32.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:32.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:58:32.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:32.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:32.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:32.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:32.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:32.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:58:32.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd4 2024-08-25T20:58:32.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T20:58:32.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:58:32.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:58:32.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: eee5a9dc-3b0d-46fe-89e9-dcc0c25140bc 2024-08-25T20:58:32.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:32.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:32.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:32.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:32.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:32.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd5 2024-08-25T20:58:32.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:32.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T20:58:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:32.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:58:32.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:32.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:32.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:32.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:32.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:32.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd6 2024-08-25T20:58:32.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:58:32.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T20:58:32.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:32.267 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:32.270 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:32.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:32.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:32.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:58:32.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:32.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:58:32.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:32.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:32.278 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:32.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:32.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:32.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:32.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:58:32.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:32.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:58:32.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:32.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:32.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:32.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:32.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:32.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:32.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-08-25T20:58:32.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:32.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:32.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:58:32.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:32.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:32.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:32.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:58:32.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:58:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:58:32.877 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:58:32.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:58:32.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:58:32.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-08-25T20:58:32.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:58:32.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:58:32.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-08-25T20:58:33.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd7 2024-08-25T20:58:33.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T20:58:33.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:58:33.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd8 2024-08-25T20:58:33.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T20:58:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:58:33.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-08-25T20:58:33.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:33.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:33.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:58:33.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:33.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:58:33.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:58:33.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:33.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:33.521 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-08-25T20:58:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:58:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: eee5a9dc-3b0d-46fe-89e9-dcc0c25140bc 2024-08-25T20:58:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T20:58:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T20:58:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:58:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:58:28' 2024-08-25T20:58:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T20:58:33.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T20:58:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:58:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T20:58:33.775 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:58:33.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:58:33.779 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:33.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:33.787 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:58:33.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:58:33.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:58:33.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:58:33.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:33.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-08-25 20:58:28' 2024-08-25T20:58:33.816 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:58:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T20:58:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:58:28' 2024-08-25T20:58:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:58:33.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:58:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T20:58:34.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 20:58:28' 2024-08-25T20:58:34.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:58:34.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:58:34.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:58:28' 2024-08-25T20:58:34.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:58:34.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T20:58:34.185 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T20:58:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T20:58:34.190 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T20:58:34.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:34.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:58:34.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary8 2024-08-25T20:58:34.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:58:34.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:58:34.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:58:34.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T20:58:34.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-08-25T20:58:34.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T20:58:34.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T20:58:34.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T20:58:34.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-08-25T20:58:35.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd9 2024-08-25T20:58:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T20:58:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T20:59:26.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=7f8c4269250db5aae669c257980987ae 2024-08-25T20:59:26.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-08-25T20:59:26.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=5df306e05f6677f8ecf4139dff5ecebd 2024-08-25T20:59:26.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-08-25T20:59:26.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 7f8c4269250db5aae669c257980987ae != \7\f\8\c\4\2\6\9\2\5\0\d\b\5\a\a\e\6\6\9\c\2\5\7\9\8\0\9\8\7\a\e ]] 2024-08-25T20:59:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 5df306e05f6677f8ecf4139dff5ecebd != \5\d\f\3\0\6\e\0\5\f\6\6\7\7\f\8\e\c\f\4\1\3\9\d\f\f\5\e\c\e\b\d ]] 2024-08-25T20:59:26.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=5f115928c88fb408a534ff321101a8af 2024-08-25T20:59:26.718 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-08-25T20:59:26.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=b718c4178e9637ca8c6af931ebd9bb3a 2024-08-25T20:59:26.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-08-25T20:59:26.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 5f115928c88fb408a534ff321101a8af != \5\f\1\1\5\9\2\8\c\8\8\f\b\4\0\8\a\5\3\4\f\f\3\2\1\1\0\1\a\8\a\f ]] 2024-08-25T20:59:26.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=b6c2d26eff95d8a35a53738e983443ce 2024-08-25T20:59:26.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-08-25T20:59:26.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ b718c4178e9637ca8c6af931ebd9bb3a != \b\7\1\8\c\4\1\7\8\e\9\6\3\7\c\a\8\c\6\a\f\9\3\1\e\b\d\9\b\b\3\a ]] 2024-08-25T20:59:27.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ b6c2d26eff95d8a35a53738e983443ce != \b\6\c\2\d\2\6\e\f\f\9\5\d\8\a\3\5\a\5\3\7\3\8\e\9\8\3\4\4\3\c\e ]] 2024-08-25T20:59:27.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=ec58ffca7d6fc410cc867fba0500a0a6 2024-08-25T20:59:27.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-08-25T20:59:27.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ ec58ffca7d6fc410cc867fba0500a0a6 != \e\c\5\8\f\f\c\a\7\d\6\f\c\4\1\0\c\c\8\6\7\f\b\a\0\5\0\0\a\0\a\6 ]] 2024-08-25T20:59:27.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:59:27.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 126084 2024-08-25T20:59:27.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:59:27.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 126086 2024-08-25T20:59:27.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:59:27.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 126088 2024-08-25T20:59:27.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=302e15489bf8458ff51f56547b8de909 2024-08-25T20:59:27.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-08-25T20:59:27.727 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 302e15489bf8458ff51f56547b8de909 != \3\0\2\e\1\5\4\8\9\b\f\8\4\5\8\f\f\5\1\f\5\6\5\4\7\b\8\d\e\9\0\9 ]] 2024-08-25T20:59:27.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:59:27.730 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 126090 2024-08-25T20:59:27.789 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=15088d778f7b8b64426a39e297aeebb7 2024-08-25T20:59:27.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-08-25T20:59:27.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 15088d778f7b8b64426a39e297aeebb7 != \1\5\0\8\8\d\7\7\8\f\7\b\8\b\6\4\4\2\6\a\3\9\e\2\9\7\a\e\e\b\b\7 ]] 2024-08-25T20:59:27.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:59:27.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 126092 2024-08-25T20:59:27.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=906c41dcb5d9c32c1a95849a7192894d 2024-08-25T20:59:27.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-08-25T20:59:28.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 906c41dcb5d9c32c1a95849a7192894d != \9\0\6\c\4\1\d\c\b\5\d\9\c\3\2\c\1\a\9\5\8\4\9\a\7\1\9\2\8\9\4\d ]] 2024-08-25T20:59:29.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=9b418e0b72721f06a92c35baa9ecf92e 2024-08-25T20:59:29.636 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-08-25T20:59:29.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 9b418e0b72721f06a92c35baa9ecf92e != \9\b\4\1\8\e\0\b\7\2\7\2\1\f\0\6\a\9\2\c\3\5\b\a\a\9\e\c\f\9\2\e ]] 2024-08-25T20:59:29.732 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:59:29.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 126094 2024-08-25T20:59:29.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:59:29.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 126096 2024-08-25T20:59:29.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:59:29.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 126098 2024-08-25T20:59:29.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T20:59:29.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 126100 2024-08-25T20:59:29.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T20:59:29.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:59:29.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T20:59:29.733 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-08-25T20:59:29.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:29.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:29.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:59:29.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:59:29.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-08-25T20:59:29.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:29.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:29.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:59:29.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:29.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:29.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:29.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:29.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:59:30.734 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-08-25T20:59:30.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:30.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:30.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:59:30.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:30.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:30.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:30.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:30.735 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:30.736 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T20:59:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T20:59:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 2579f173-30f9-4673-a851-61403f4bd16d 2024-08-25T20:59:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:59:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3334963.2,"bytes_per_snapshot":66699264.0,"last_snapshot_bytes":66699264,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"} 2024-08-25T20:59:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:59:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:59:28' 2024-08-25T20:59:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:59:30.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:59:30.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:59:30.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3334963.2,"bytes_per_snapshot":66699264.0,"last_snapshot_bytes":66699264,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"} 2024-08-25T20:59:30.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:59:30.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:30.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:59:30.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:30.827 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:59:30.829 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:59:30.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:59:30.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:59:30.834 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-08-25 20:59:28' 2024-08-25T20:59:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:59:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:59:28' 2024-08-25T20:59:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:30.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:30.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:59:30.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary12579f173-30f9-4673-a851-61403f4bd16dup+replayingreplaying, {"bytes_per_second":3334963.2,"bytes_per_snapshot":66699264.0,"last_snapshot_bytes":66699264,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+replayingreplaying, {"bytes_per_second":3315916.8,"bytes_per_snapshot":66318336.0,"last_snapshot_bytes":66318336,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+replayingreplaying, {"bytes_per_second":3346636.8,"bytes_per_snapshot":66932736.0,"last_snapshot_bytes":66932736,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+replayingreplaying, {"bytes_per_second":3417907.2,"bytes_per_snapshot":68358144.0,"last_snapshot_bytes":68358144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+replayingreplaying, {"bytes_per_second":3322675.2,"bytes_per_snapshot":66453504.0,"last_snapshot_bytes":66453504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+replayingreplaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+replayingreplaying, {"bytes_per_second":3503718.4,"bytes_per_snapshot":70074368.0,"last_snapshot_bytes":70074368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+replayingreplaying, {"bytes_per_second":3179315.2,"bytes_per_snapshot":63586304.0,"last_snapshot_bytes":63586304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+replayingreplaying, {"bytes_per_second":3332915.2,"bytes_per_snapshot":66658304.0,"last_snapshot_bytes":66658304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619514,"remote_snapshot_timestamp":1724619514,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+replayingreplaying, {"bytes_per_second":3265126.4,"bytes_per_snapshot":65302528.0,"last_snapshot_bytes":65302528,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28' 2024-08-25T20:59:30.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:59:30.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T20:59:30.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:30.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T20:59:30.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:59:30.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:30.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:59:30.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:30.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:30.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:30.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary1 2024-08-25T20:59:30.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:59:30.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:30.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:59:30.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-08-25T20:59:31.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-08-25T20:59:31.198 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:31.197+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adf720b60 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:59:31.202 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:31.197+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973680 [3/2579f173-30f9-4673-a851-61403f4bd16d] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:59:33.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:59:33.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-08-25T20:59:33.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:33.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:33.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T20:59:33.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:33.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:59:33.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary1 2024-08-25T20:59:33.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:33.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary1 2024-08-25T20:59:33.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:33.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:33.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:33.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:59:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-08-25T20:59:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:59:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:34.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:34.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T20:59:34.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T20:59:34.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 4d392b83-7305-4b60-b6ea-84b62c29ac4a 2024-08-25T20:59:34.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:59:34.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3346636.8,"bytes_per_snapshot":66932736.0,"last_snapshot_bytes":66932736,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"} 2024-08-25T20:59:34.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:59:34.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:59:28' 2024-08-25T20:59:34.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:59:34.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:59:34.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:59:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3346636.8,"bytes_per_snapshot":66932736.0,"last_snapshot_bytes":66932736,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"} 2024-08-25T20:59:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:59:34.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:34.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:59:34.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:34.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:59:34.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:59:34.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:59:34.206 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:59:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-08-25 20:59:28' 2024-08-25T20:59:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:34.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:34.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:59:34.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:34.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:59:28' 2024-08-25T20:59:34.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:34.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:34.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:59:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING941904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+replayingreplaying, {"bytes_per_second":3315916.8,"bytes_per_snapshot":66318336.0,"last_snapshot_bytes":66318336,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary24d392b83-7305-4b60-b6ea-84b62c29ac4aup+replayingreplaying, {"bytes_per_second":3346636.8,"bytes_per_snapshot":66932736.0,"last_snapshot_bytes":66932736,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+replayingreplaying, {"bytes_per_second":3417907.2,"bytes_per_snapshot":68358144.0,"last_snapshot_bytes":68358144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+replayingreplaying, {"bytes_per_second":3322675.2,"bytes_per_snapshot":66453504.0,"last_snapshot_bytes":66453504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+replayingreplaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+replayingreplaying, {"bytes_per_second":3503718.4,"bytes_per_snapshot":70074368.0,"last_snapshot_bytes":70074368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+replayingreplaying, {"bytes_per_second":3179315.2,"bytes_per_snapshot":63586304.0,"last_snapshot_bytes":63586304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+replayingreplaying, {"bytes_per_second":3332915.2,"bytes_per_snapshot":66658304.0,"last_snapshot_bytes":66658304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619514,"remote_snapshot_timestamp":1724619514,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+replayingreplaying, {"bytes_per_second":3265126.4,"bytes_per_snapshot":65302528.0,"last_snapshot_bytes":65302528,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28' 2024-08-25T20:59:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:59:34.318 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T20:59:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T20:59:34.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:59:34.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:34.327 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:59:34.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary2 2024-08-25T20:59:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:59:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:59:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-08-25T20:59:34.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-08-25T20:59:34.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:34.529+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3c19040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:59:34.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:34.529+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972280 [3/4d392b83-7305-4b60-b6ea-84b62c29ac4a] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:59:36.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:59:36.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-08-25T20:59:36.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:36.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:36.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T20:59:36.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:36.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:59:36.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary2 2024-08-25T20:59:36.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:36.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary2 2024-08-25T20:59:36.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:59:36.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T20:59:36.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-08-25T20:59:36.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:36.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:36.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:59:36.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:59:36.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-08-25T20:59:36.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:36.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:36.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:59:36.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:36.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:36.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:36.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:36.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:59:37.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-08-25T20:59:37.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:37.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:37.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:59:37.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:37.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:37.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:37.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:37.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:37.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T20:59:37.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T20:59:37.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9178aa8-3d6a-41d7-946b-0dc934500683 2024-08-25T20:59:37.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:59:37.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3417907.2,"bytes_per_snapshot":68358144.0,"last_snapshot_bytes":68358144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"} 2024-08-25T20:59:37.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:59:37.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:59:28' 2024-08-25T20:59:37.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:59:37.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:59:37.199 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:59:37.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3417907.2,"bytes_per_snapshot":68358144.0,"last_snapshot_bytes":68358144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"} 2024-08-25T20:59:37.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:59:37.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:37.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:59:37.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:37.205 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:59:37.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:59:37.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:59:37.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:59:37.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:37.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-08-25 20:59:28' 2024-08-25T20:59:37.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:37.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:37.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:59:37.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:37.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:59:28' 2024-08-25T20:59:37.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:37.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:37.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:59:37.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING841904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+replayingreplaying, {"bytes_per_second":3315916.8,"bytes_per_snapshot":66318336.0,"last_snapshot_bytes":66318336,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary3b9178aa8-3d6a-41d7-946b-0dc934500683up+replayingreplaying, {"bytes_per_second":3417907.2,"bytes_per_snapshot":68358144.0,"last_snapshot_bytes":68358144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+replayingreplaying, {"bytes_per_second":3322675.2,"bytes_per_snapshot":66453504.0,"last_snapshot_bytes":66453504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+replayingreplaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+replayingreplaying, {"bytes_per_second":3503718.4,"bytes_per_snapshot":70074368.0,"last_snapshot_bytes":70074368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+replayingreplaying, {"bytes_per_second":3179315.2,"bytes_per_snapshot":63586304.0,"last_snapshot_bytes":63586304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+replayingreplaying, {"bytes_per_second":3332915.2,"bytes_per_snapshot":66658304.0,"last_snapshot_bytes":66658304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619514,"remote_snapshot_timestamp":1724619514,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+replayingreplaying, {"bytes_per_second":3265126.4,"bytes_per_snapshot":65302528.0,"last_snapshot_bytes":65302528,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28' 2024-08-25T20:59:37.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:59:37.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T20:59:37.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:37.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T20:59:37.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:59:37.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:37.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:59:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary3 2024-08-25T20:59:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:59:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:37.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:59:37.333 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-08-25T20:59:37.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-08-25T20:59:37.540 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:37.539+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4fca9c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:59:37.540 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:37.539+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973180 [3/b9178aa8-3d6a-41d7-946b-0dc934500683] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:59:39.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:59:39.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-08-25T20:59:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T20:59:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:59:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary3 2024-08-25T20:59:39.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:39.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary3 2024-08-25T20:59:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:59:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T20:59:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-08-25T20:59:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:59:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:59:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-08-25T20:59:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:59:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:39.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:39.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:39.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:39.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:59:40.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-08-25T20:59:40.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:40.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:59:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T20:59:40.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T20:59:40.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 890c934e-762e-4439-a4cf-97e4d48d8c7d 2024-08-25T20:59:40.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:59:40.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3322675.2,"bytes_per_snapshot":66453504.0,"last_snapshot_bytes":66453504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"} 2024-08-25T20:59:40.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:59:40.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:59:28' 2024-08-25T20:59:40.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:59:40.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:59:40.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:59:40.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3322675.2,"bytes_per_snapshot":66453504.0,"last_snapshot_bytes":66453504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"} 2024-08-25T20:59:40.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:59:40.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:40.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:59:40.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:40.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:59:40.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:59:40.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:59:40.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:59:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-08-25 20:59:28' 2024-08-25T20:59:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:40.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:59:40.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:40.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:59:28' 2024-08-25T20:59:40.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:40.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:40.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:59:40.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING741904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+replayingreplaying, {"bytes_per_second":3315916.8,"bytes_per_snapshot":66318336.0,"last_snapshot_bytes":66318336,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary4890c934e-762e-4439-a4cf-97e4d48d8c7dup+replayingreplaying, {"bytes_per_second":3322675.2,"bytes_per_snapshot":66453504.0,"last_snapshot_bytes":66453504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+replayingreplaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+replayingreplaying, {"bytes_per_second":3503718.4,"bytes_per_snapshot":70074368.0,"last_snapshot_bytes":70074368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+replayingreplaying, {"bytes_per_second":3179315.2,"bytes_per_snapshot":63586304.0,"last_snapshot_bytes":63586304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+replayingreplaying, {"bytes_per_second":3332915.2,"bytes_per_snapshot":66658304.0,"last_snapshot_bytes":66658304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619514,"remote_snapshot_timestamp":1724619514,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+replayingreplaying, {"bytes_per_second":3265126.4,"bytes_per_snapshot":65302528.0,"last_snapshot_bytes":65302528,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28' 2024-08-25T20:59:40.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:59:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T20:59:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:40.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T20:59:40.328 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:59:40.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:40.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:59:40.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary4 2024-08-25T20:59:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:59:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:59:40.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-08-25T20:59:40.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-08-25T20:59:40.528 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:40.527+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3388ea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:59:40.529 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:40.527+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972500 [3/890c934e-762e-4439-a4cf-97e4d48d8c7d] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:59:41.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:59:41.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-08-25T20:59:41.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:41.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:41.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T20:59:41.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:41.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:59:41.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary4 2024-08-25T20:59:41.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:41.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary4 2024-08-25T20:59:41.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:59:41.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:59:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-08-25T20:59:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:59:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:42.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:42.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:42.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:42.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T20:59:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T20:59:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 723168d9-bb4b-4d32-a3d8-99d77786ac4b 2024-08-25T20:59:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:59:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"} 2024-08-25T20:59:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:59:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:59:28' 2024-08-25T20:59:43.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:59:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:59:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:59:43.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"} 2024-08-25T20:59:43.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:59:43.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:43.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:59:43.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:43.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:59:43.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:59:43.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:59:43.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:59:43.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:43.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-08-25 20:59:28' 2024-08-25T20:59:43.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:43.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:43.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:59:43.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:43.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:59:28' 2024-08-25T20:59:43.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:43.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:43.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:59:43.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING641904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+replayingreplaying, {"bytes_per_second":3315916.8,"bytes_per_snapshot":66318336.0,"last_snapshot_bytes":66318336,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary5723168d9-bb4b-4d32-a3d8-99d77786ac4bup+replayingreplaying, {"bytes_per_second":3338240.0,"bytes_per_snapshot":66764800.0,"last_snapshot_bytes":66764800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+replayingreplaying, {"bytes_per_second":3503718.4,"bytes_per_snapshot":70074368.0,"last_snapshot_bytes":70074368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+replayingreplaying, {"bytes_per_second":3179315.2,"bytes_per_snapshot":63586304.0,"last_snapshot_bytes":63586304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+replayingreplaying, {"bytes_per_second":3332915.2,"bytes_per_snapshot":66658304.0,"last_snapshot_bytes":66658304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619514,"remote_snapshot_timestamp":1724619514,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+replayingreplaying, {"bytes_per_second":3265126.4,"bytes_per_snapshot":65302528.0,"last_snapshot_bytes":65302528,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28' 2024-08-25T20:59:43.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:59:43.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T20:59:43.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:43.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T20:59:43.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:59:43.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:43.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:59:43.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary5 2024-08-25T20:59:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:59:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:59:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-08-25T20:59:43.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-08-25T20:59:43.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:43.370+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3389a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:59:43.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:43.370+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972780 [3/723168d9-bb4b-4d32-a3d8-99d77786ac4b] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:59:44.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:59:44.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-08-25T20:59:44.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:44.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:44.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T20:59:44.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:44.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:59:44.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary5 2024-08-25T20:59:44.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:44.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary5 2024-08-25T20:59:44.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:59:44.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T20:59:44.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-08-25T20:59:44.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:44.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:44.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:59:44.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:59:44.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-08-25T20:59:44.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:44.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:44.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:59:44.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:44.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:44.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:44.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:44.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:59:45.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-08-25T20:59:45.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:45.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:45.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:59:45.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:45.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:45.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:45.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:45.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:45.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T20:59:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T20:59:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ffdc83df-6651-4668-9561-234d384d3d02 2024-08-25T20:59:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:59:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3503718.4,"bytes_per_snapshot":70074368.0,"last_snapshot_bytes":70074368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"} 2024-08-25T20:59:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:59:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:59:28' 2024-08-25T20:59:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:59:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:59:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:59:46.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3503718.4,"bytes_per_snapshot":70074368.0,"last_snapshot_bytes":70074368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"} 2024-08-25T20:59:46.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:59:46.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:46.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:59:46.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:46.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:59:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:59:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:59:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:59:46.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:46.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-08-25 20:59:28' 2024-08-25T20:59:46.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:46.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:46.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:59:46.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:46.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:59:28' 2024-08-25T20:59:46.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:46.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:46.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:59:46.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING541904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+replayingreplaying, {"bytes_per_second":3315916.8,"bytes_per_snapshot":66318336.0,"last_snapshot_bytes":66318336,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary6ffdc83df-6651-4668-9561-234d384d3d02up+replayingreplaying, {"bytes_per_second":3503718.4,"bytes_per_snapshot":70074368.0,"last_snapshot_bytes":70074368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619512,"remote_snapshot_timestamp":1724619512,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+replayingreplaying, {"bytes_per_second":3179315.2,"bytes_per_snapshot":63586304.0,"last_snapshot_bytes":63586304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+replayingreplaying, {"bytes_per_second":3332915.2,"bytes_per_snapshot":66658304.0,"last_snapshot_bytes":66658304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619514,"remote_snapshot_timestamp":1724619514,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+replayingreplaying, {"bytes_per_second":3265126.4,"bytes_per_snapshot":65302528.0,"last_snapshot_bytes":65302528,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28' 2024-08-25T20:59:46.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:59:46.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T20:59:46.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:46.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T20:59:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:59:46.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:46.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:59:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary6 2024-08-25T20:59:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:59:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:59:46.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-08-25T20:59:46.219 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-08-25T20:59:46.368 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:46.367+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3662680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:59:46.368 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:46.367+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972c80 [3/ffdc83df-6651-4668-9561-234d384d3d02] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:59:47.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:59:47.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-08-25T20:59:47.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:47.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:47.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T20:59:47.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:47.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:59:47.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary6 2024-08-25T20:59:47.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:47.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary6 2024-08-25T20:59:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:59:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T20:59:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-08-25T20:59:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:59:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:59:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-08-25T20:59:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:47.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:59:47.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:47.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:47.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:47.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:47.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:59:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-08-25T20:59:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:59:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:48.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:48.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:48.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:48.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T20:59:49.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T20:59:49.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 393b7e43-14d2-4eab-bbbb-53da05c7fcc6 2024-08-25T20:59:49.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:59:49.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3179315.2,"bytes_per_snapshot":63586304.0,"last_snapshot_bytes":63586304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"} 2024-08-25T20:59:49.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:59:49.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:59:28' 2024-08-25T20:59:49.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:59:49.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:59:49.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:59:49.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3179315.2,"bytes_per_snapshot":63586304.0,"last_snapshot_bytes":63586304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"} 2024-08-25T20:59:49.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:59:49.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:49.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:59:49.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:49.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:59:49.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:59:49.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:59:49.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:59:49.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:49.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-08-25 20:59:28' 2024-08-25T20:59:49.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:49.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:49.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:59:49.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:49.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:59:28' 2024-08-25T20:59:49.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:49.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:49.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:59:49.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING441904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+replayingreplaying, {"bytes_per_second":3315916.8,"bytes_per_snapshot":66318336.0,"last_snapshot_bytes":66318336,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary7393b7e43-14d2-4eab-bbbb-53da05c7fcc6up+replayingreplaying, {"bytes_per_second":3179315.2,"bytes_per_snapshot":63586304.0,"last_snapshot_bytes":63586304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619511,"remote_snapshot_timestamp":1724619511,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+replayingreplaying, {"bytes_per_second":3332915.2,"bytes_per_snapshot":66658304.0,"last_snapshot_bytes":66658304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619514,"remote_snapshot_timestamp":1724619514,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+replayingreplaying, {"bytes_per_second":3265126.4,"bytes_per_snapshot":65302528.0,"last_snapshot_bytes":65302528,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28' 2024-08-25T20:59:49.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:59:49.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T20:59:49.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:49.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T20:59:49.154 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:59:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:59:49.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:49.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:49.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:49.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary7 2024-08-25T20:59:49.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:59:49.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:49.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:59:49.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-08-25T20:59:49.218 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-08-25T20:59:49.358 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:49.358+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adf01bd40 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:59:49.359 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:49.358+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ae365f900 [3/393b7e43-14d2-4eab-bbbb-53da05c7fcc6] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:59:50.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:59:50.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-08-25T20:59:50.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:50.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:50.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T20:59:50.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:50.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:59:50.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary7 2024-08-25T20:59:50.937 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:50.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary7 2024-08-25T20:59:50.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:59:50.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T20:59:50.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-08-25T20:59:50.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:59:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:59:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-08-25T20:59:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:59:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:59:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-08-25T20:59:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:59:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T20:59:52.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T20:59:52.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: eee5a9dc-3b0d-46fe-89e9-dcc0c25140bc 2024-08-25T20:59:52.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:59:52.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3332915.2,"bytes_per_snapshot":66658304.0,"last_snapshot_bytes":66658304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619514,"remote_snapshot_timestamp":1724619514,"replay_state":"idle"} 2024-08-25T20:59:52.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:59:52.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:59:28' 2024-08-25T20:59:52.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:59:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:59:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:59:52.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3332915.2,"bytes_per_snapshot":66658304.0,"last_snapshot_bytes":66658304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619514,"remote_snapshot_timestamp":1724619514,"replay_state":"idle"} 2024-08-25T20:59:52.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:59:52.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:52.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:59:52.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:52.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:59:52.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:59:52.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:59:52.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:59:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-08-25 20:59:28' 2024-08-25T20:59:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:59:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:59:28' 2024-08-25T20:59:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:52.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:59:52.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING341904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+replayingreplaying, {"bytes_per_second":3315916.8,"bytes_per_snapshot":66318336.0,"last_snapshot_bytes":66318336,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary8eee5a9dc-3b0d-46fe-89e9-dcc0c25140bcup+replayingreplaying, {"bytes_per_second":3332915.2,"bytes_per_snapshot":66658304.0,"last_snapshot_bytes":66658304,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619514,"remote_snapshot_timestamp":1724619514,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+replayingreplaying, {"bytes_per_second":3265126.4,"bytes_per_snapshot":65302528.0,"last_snapshot_bytes":65302528,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28' 2024-08-25T20:59:52.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:59:52.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T20:59:52.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:52.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T20:59:52.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:59:52.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:52.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:59:52.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:52.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:52.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:52.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary8 2024-08-25T20:59:52.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:59:52.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:52.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:59:52.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-08-25T20:59:52.200 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-08-25T20:59:52.343 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:52.342+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4c3a340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:59:52.343 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:52.342+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ae365fb80 [3/eee5a9dc-3b0d-46fe-89e9-dcc0c25140bc] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:59:53.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:59:53.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-08-25T20:59:53.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T20:59:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:59:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary8 2024-08-25T20:59:53.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:53.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary8 2024-08-25T20:59:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:53.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:59:54.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-08-25T20:59:54.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:54.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:54.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:59:54.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:54.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:54.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:54.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:54.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:54.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T20:59:55.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T20:59:55.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 2e55cf60-b17c-4f9f-8466-e6c4722a0dd6 2024-08-25T20:59:55.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:59:55.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3265126.4,"bytes_per_snapshot":65302528.0,"last_snapshot_bytes":65302528,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"} 2024-08-25T20:59:55.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:59:55.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:59:28' 2024-08-25T20:59:55.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:59:55.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:59:55.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:59:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3265126.4,"bytes_per_snapshot":65302528.0,"last_snapshot_bytes":65302528,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"} 2024-08-25T20:59:55.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:59:55.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:55.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:59:55.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:55.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:59:55.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:59:55.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:59:55.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:59:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-08-25 20:59:28' 2024-08-25T20:59:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:59:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:59:28' 2024-08-25T20:59:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:55.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:59:55.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING241904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+replayingreplaying, {"bytes_per_second":3315916.8,"bytes_per_snapshot":66318336.0,"last_snapshot_bytes":66318336,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28image-primary92e55cf60-b17c-4f9f-8466-e6c4722a0dd6up+replayingreplaying, {"bytes_per_second":3265126.4,"bytes_per_snapshot":65302528.0,"last_snapshot_bytes":65302528,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28' 2024-08-25T20:59:55.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:59:55.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T20:59:55.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:55.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T20:59:55.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:59:55.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:55.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:59:55.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:55.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:55.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:55.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary9 2024-08-25T20:59:55.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:59:55.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:55.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:59:55.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-08-25T20:59:55.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-08-25T20:59:55.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:55.374+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3388820 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:59:55.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:55.374+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade973b80 [3/2e55cf60-b17c-4f9f-8466-e6c4722a0dd6] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:59:56.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T20:59:56.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-08-25T20:59:56.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:56.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:56.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T20:59:56.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:56.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T20:59:56.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary9 2024-08-25T20:59:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary9 2024-08-25T20:59:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T20:59:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T20:59:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-08-25T20:59:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:59:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T20:59:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-08-25T20:59:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:59:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T20:59:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T20:59:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-08-25T20:59:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:59:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:57.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T20:59:57.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T20:59:57.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:57.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:57.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T20:59:58.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T20:59:58.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 36517776-fbdb-47e2-b79b-76a572c69ca6 2024-08-25T20:59:58.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T20:59:58.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3315916.8,"bytes_per_snapshot":66318336.0,"last_snapshot_bytes":66318336,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"} 2024-08-25T20:59:58.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T20:59:58.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 20:59:28' 2024-08-25T20:59:58.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T20:59:58.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T20:59:58.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T20:59:58.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3315916.8,"bytes_per_snapshot":66318336.0,"last_snapshot_bytes":66318336,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"} 2024-08-25T20:59:58.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T20:59:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T20:59:58.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:58.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T20:59:58.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T20:59:58.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T20:59:58.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T20:59:58.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:58.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-08-25 20:59:28' 2024-08-25T20:59:58.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T20:59:58.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:58.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:59:58.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T20:59:58.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 20:59:28' 2024-08-25T20:59:58.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T20:59:58.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T20:59:58.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T20:59:58.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING141904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1036517776-fbdb-47e2-b79b-76a572c69ca6up+replayingreplaying, {"bytes_per_second":3315916.8,"bytes_per_snapshot":66318336.0,"last_snapshot_bytes":66318336,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619510,"remote_snapshot_timestamp":1724619510,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 20:59:28' 2024-08-25T20:59:58.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T20:59:58.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T20:59:58.161 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 20:59:28' 2024-08-25T20:59:58.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T20:59:58.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T20:59:58.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T20:59:58.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T20:59:58.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T20:59:58.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:58.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T20:59:58.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary10 2024-08-25T20:59:58.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T20:59:58.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T20:59:58.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T20:59:58.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-08-25T20:59:58.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-08-25T20:59:58.340 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:58.339+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8bb380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:59:58.340 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:58.339+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ae4fe2a00 [3/36517776-fbdb-47e2-b79b-76a572c69ca6] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T20:59:58.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:58.373+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade886ea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T20:59:58.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T20:59:58.373+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ae4fe2a00 [3/36517776-fbdb-47e2-b79b-76a572c69ca6] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T21:00:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T21:00:00.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-08-25T21:00:00.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:00.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:00.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:00.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:00:00.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T21:00:00.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary10 2024-08-25T21:00:00.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:00:00.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary10 2024-08-25T21:00:00.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..10} 2024-08-25T21:00:00.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS=() 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS=() 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS=() 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS=() 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ RET=0 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary1 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:00.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T21:00:00.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T21:00:00.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:00.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-08-25T21:00:00.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T21:00:00.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:00.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T21:00:00.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:00:00.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-08-25T21:00:00.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:00.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:00.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:00.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:00.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-08-25T21:00:00.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T21:00:00.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-08-25T21:00:00.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary1': 2024-08-25T21:00:00.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T21:00:00.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T21:00:00.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T21:00:00.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3d8ac02bb78f 2024-08-25T21:00:00.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3d8ac02bb78f 2024-08-25T21:00:00.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T21:00:00.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T21:00:00.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T21:00:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T21:00:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 21:00:00 2024 2024-08-25T21:00:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 21:00:00 2024 2024-08-25T21:00:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 21:00:00 2024 2024-08-25T21:00:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T21:00:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T21:00:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: aadbded2-2750-407c-895a-1bb2e92d5a94 2024-08-25T21:00:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T21:00:00.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:00:00.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:00:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-08-25T21:00:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd0 2024-08-25T21:00:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T21:00:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-08-25T21:00:01.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T21:00:01.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T21:00:01.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T21:00:01.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 563c23f1-640b-410e-a8f6-b177fe8ccc1a 2024-08-25T21:00:01.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T21:00:01.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T21:00:01.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:01.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T21:00:01.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T21:00:01.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T21:00:01.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T21:00:01.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:01.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary1 2024-08-25T21:00:01.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-08-25T21:00:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T21:00:01.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary1 2024-08-25T21:00:01.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T21:00:01.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-08-25T21:00:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T21:00:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:00:01.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary1 2024-08-25T21:00:01.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T21:00:01.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary2 2024-08-25T21:00:01.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-08-25T21:00:01.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary1 2024-08-25T21:00:01.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:01.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:01.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-08-25T21:00:01.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:01.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:01.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T21:00:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-08-25T21:00:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T21:00:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T21:00:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T21:00:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-08-25T21:00:01.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T21:00:01.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:01.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T21:00:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:00:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-08-25T21:00:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-08-25T21:00:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T21:00:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-08-25T21:00:01.817 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T21:00:01.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2974 2024-08-25T21:00:01.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-08-25T21:00:01.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary2': 2024-08-25T21:00:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T21:00:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T21:00:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T21:00:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3d99f46f6496 2024-08-25T21:00:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3d99f46f6496 2024-08-25T21:00:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T21:00:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T21:00:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T21:00:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T21:00:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 21:00:01 2024 2024-08-25T21:00:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 21:00:01 2024 2024-08-25T21:00:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 21:00:01 2024 2024-08-25T21:00:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T21:00:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T21:00:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: b9ffbd30-cb37-47e1-9712-592083b16894 2024-08-25T21:00:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T21:00:01.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:00:01.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:00:01.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-08-25T21:00:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd1 2024-08-25T21:00:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T21:00:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-08-25T21:00:02.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T21:00:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T21:00:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T21:00:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: a448a6fd-e30b-4bcc-a378-268bf1ce57d5 2024-08-25T21:00:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T21:00:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T21:00:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T21:00:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T21:00:02.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T21:00:02.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T21:00:02.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary2 2024-08-25T21:00:02.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-08-25T21:00:02.174 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T21:00:02.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary2 2024-08-25T21:00:02.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T21:00:02.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-08-25T21:00:02.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:02.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:02.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:02.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T21:00:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:00:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T21:00:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary3 2024-08-25T21:00:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-08-25T21:00:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary2 2024-08-25T21:00:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T21:00:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-08-25T21:00:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary2 2024-08-25T21:00:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-08-25T21:00:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T21:00:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T21:00:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T21:00:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-08-25T21:00:02.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T21:00:02.272 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:02.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T21:00:02.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:00:02.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-08-25T21:00:02.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:02.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:02.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:02.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:02.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-08-25T21:00:02.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T21:00:02.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-08-25T21:00:02.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T21:00:02.823 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2976 2024-08-25T21:00:02.832 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:02.833 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-08-25T21:00:02.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary3': 2024-08-25T21:00:02.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T21:00:02.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T21:00:02.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T21:00:02.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3da8f08b010b 2024-08-25T21:00:02.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3da8f08b010b 2024-08-25T21:00:02.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T21:00:02.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T21:00:02.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T21:00:02.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T21:00:02.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 21:00:02 2024 2024-08-25T21:00:02.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 21:00:02 2024 2024-08-25T21:00:02.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 21:00:02 2024 2024-08-25T21:00:02.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T21:00:02.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T21:00:02.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 5eac9715-6778-491b-b159-ca14ad42f707 2024-08-25T21:00:02.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T21:00:02.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:00:02.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:00:02.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-08-25T21:00:03.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd2 2024-08-25T21:00:03.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T21:00:03.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-08-25T21:00:03.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T21:00:03.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T21:00:03.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T21:00:03.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: e2b3e792-73df-4e8d-98c2-b8a8f1287545 2024-08-25T21:00:03.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T21:00:03.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T21:00:03.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:03.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T21:00:03.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T21:00:03.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T21:00:03.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T21:00:03.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary3 2024-08-25T21:00:03.124 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-08-25T21:00:03.184 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T21:00:03.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary3 2024-08-25T21:00:03.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T21:00:03.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-08-25T21:00:03.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:03.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:03.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:03.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary3 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary4 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary3 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:03.211 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:03.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:03.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T21:00:03.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T21:00:03.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T21:00:03.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:03.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-08-25T21:00:03.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T21:00:03.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:03.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T21:00:03.283 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:00:03.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-08-25T21:00:03.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:03.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:03.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:03.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:03.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-08-25T21:00:03.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T21:00:03.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-08-25T21:00:03.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2978 2024-08-25T21:00:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T21:00:03.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:03.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-08-25T21:00:03.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary4': 2024-08-25T21:00:03.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T21:00:03.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T21:00:03.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T21:00:03.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3db7b2b481 2024-08-25T21:00:03.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3db7b2b481 2024-08-25T21:00:03.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T21:00:03.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T21:00:03.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T21:00:03.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T21:00:03.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 21:00:03 2024 2024-08-25T21:00:03.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 21:00:03 2024 2024-08-25T21:00:03.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 21:00:03 2024 2024-08-25T21:00:03.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T21:00:03.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T21:00:03.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: a6d3b19e-9e6d-435d-9a57-070b64a2d592 2024-08-25T21:00:03.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T21:00:03.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:00:03.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:00:03.913 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-08-25T21:00:04.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd3 2024-08-25T21:00:04.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T21:00:04.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-08-25T21:00:04.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T21:00:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T21:00:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T21:00:04.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 4bafd372-57e9-43e3-ae7d-bca985f66649 2024-08-25T21:00:04.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T21:00:04.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T21:00:04.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:04.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T21:00:04.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T21:00:04.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T21:00:04.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T21:00:04.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:04.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary4 2024-08-25T21:00:04.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-08-25T21:00:04.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T21:00:04.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary4 2024-08-25T21:00:04.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T21:00:04.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-08-25T21:00:04.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:04.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:04.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:04.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:04.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:04.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:04.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:04.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:04.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:04.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T21:00:04.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary4 2024-08-25T21:00:04.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:00:04.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T21:00:04.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary5 2024-08-25T21:00:04.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-08-25T21:00:04.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary4 2024-08-25T21:00:04.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-08-25T21:00:04.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:04.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:04.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:04.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:04.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:04.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:04.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-08-25T21:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T21:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T21:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-08-25T21:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-08-25T21:00:04.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T21:00:04.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-08-25T21:00:04.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:04.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:04.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:04.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:04.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:04.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:04.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T21:00:04.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2980 2024-08-25T21:00:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-08-25T21:00:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:04.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary5': 2024-08-25T21:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T21:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T21:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T21:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3dc6f66d9f35 2024-08-25T21:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3dc6f66d9f35 2024-08-25T21:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T21:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T21:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T21:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T21:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 21:00:04 2024 2024-08-25T21:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 21:00:04 2024 2024-08-25T21:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 21:00:04 2024 2024-08-25T21:00:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T21:00:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T21:00:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: ad28100c-6a8b-4e3f-9d4a-e05c1aff7120 2024-08-25T21:00:04.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T21:00:04.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:00:04.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:00:04.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-08-25T21:00:05.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd4 2024-08-25T21:00:05.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T21:00:05.158 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-08-25T21:00:05.178 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T21:00:05.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T21:00:05.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T21:00:05.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 3f3668b9-193b-417a-aad3-ecddfa3db6b3 2024-08-25T21:00:05.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T21:00:05.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T21:00:05.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:05.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T21:00:05.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T21:00:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T21:00:05.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T21:00:05.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:05.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary5 2024-08-25T21:00:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-08-25T21:00:05.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T21:00:05.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary5 2024-08-25T21:00:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T21:00:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-08-25T21:00:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:00:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:05.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:05.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T21:00:05.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:00:05.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:00:05.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T21:00:05.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary6 2024-08-25T21:00:05.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-08-25T21:00:05.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary5 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary5 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:05.422 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:05.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:05.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:05.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-08-25T21:00:05.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:05.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:05.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T21:00:05.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T21:00:05.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T21:00:05.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:05.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-08-25T21:00:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T21:00:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T21:00:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:00:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-08-25T21:00:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-08-25T21:00:05.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T21:00:05.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-08-25T21:00:05.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:05.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:05.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:05.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:05.835 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:05.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:05.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T21:00:05.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2982 2024-08-25T21:00:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:05.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-08-25T21:00:05.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2983 2024-08-25T21:00:05.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary6': 2024-08-25T21:00:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T21:00:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3dd7ed62e98 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3dd7ed62e98 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 21:00:05 2024 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 21:00:05 2024 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 21:00:05 2024 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 547a99b7-e2b8-4de2-9d6e-c0c889ebf727 2024-08-25T21:00:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T21:00:06.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:00:06.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:00:06.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-08-25T21:00:06.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd5 2024-08-25T21:00:06.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T21:00:06.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-08-25T21:00:06.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T21:00:06.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T21:00:06.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T21:00:06.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 43e30baa-8a4d-4918-818b-589024c81810 2024-08-25T21:00:06.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T21:00:06.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T21:00:06.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:06.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T21:00:06.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T21:00:06.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T21:00:06.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T21:00:06.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:06.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary6 2024-08-25T21:00:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-08-25T21:00:06.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T21:00:06.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary6 2024-08-25T21:00:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T21:00:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:00:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary7 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary6 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary6 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-08-25T21:00:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T21:00:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T21:00:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T21:00:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-08-25T21:00:06.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T21:00:06.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:06.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T21:00:06.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:00:06.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-08-25T21:00:06.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:06.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:06.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:06.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:06.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-08-25T21:00:06.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:06.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:06.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:06.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:06.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:06.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:06.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2986 2024-08-25T21:00:06.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2985 2024-08-25T21:00:06.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T21:00:06.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-08-25T21:00:06.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:06.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:07.836 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2988 2024-08-25T21:00:07.837 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T21:00:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:07.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-08-25T21:00:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary7': 2024-08-25T21:00:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T21:00:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T21:00:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T21:00:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3de64c6ff8d5 2024-08-25T21:00:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3de64c6ff8d5 2024-08-25T21:00:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T21:00:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T21:00:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T21:00:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T21:00:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 21:00:06 2024 2024-08-25T21:00:07.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 21:00:06 2024 2024-08-25T21:00:07.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 21:00:06 2024 2024-08-25T21:00:07.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T21:00:07.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T21:00:07.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 9de1696e-ed4d-41f7-8c4e-02953f84c7dc 2024-08-25T21:00:07.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T21:00:07.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:00:07.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:00:07.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-08-25T21:00:08.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd6 2024-08-25T21:00:08.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T21:00:08.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-08-25T21:00:08.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T21:00:08.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T21:00:08.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T21:00:08.202 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: a38abb97-2aa2-475c-a555-8608b78b1f50 2024-08-25T21:00:08.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T21:00:08.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T21:00:08.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:08.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T21:00:08.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T21:00:08.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T21:00:08.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T21:00:08.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary7 2024-08-25T21:00:08.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-08-25T21:00:08.329 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T21:00:08.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary7 2024-08-25T21:00:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T21:00:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-08-25T21:00:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:00:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:08.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:08.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:08.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:00:08.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T21:00:08.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:00:08.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary7 2024-08-25T21:00:08.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T21:00:08.358 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary8 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary7 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-08-25T21:00:08.359 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T21:00:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T21:00:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T21:00:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-08-25T21:00:08.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T21:00:08.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:08.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T21:00:08.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:00:08.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-08-25T21:00:08.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:08.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:08.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:08.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:08.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-08-25T21:00:08.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T21:00:08.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-08-25T21:00:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T21:00:08.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2990 2024-08-25T21:00:08.860 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2989 2024-08-25T21:00:08.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-08-25T21:00:08.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:08.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:08.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:08.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:00:08.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:08.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:08.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:08.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:00:08.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary8': 2024-08-25T21:00:08.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T21:00:08.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T21:00:08.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T21:00:08.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3dfa723eef7e 2024-08-25T21:00:08.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3dfa723eef7e 2024-08-25T21:00:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T21:00:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T21:00:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T21:00:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T21:00:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 21:00:08 2024 2024-08-25T21:00:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 21:00:08 2024 2024-08-25T21:00:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 21:00:08 2024 2024-08-25T21:00:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T21:00:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T21:00:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: abb0fd4b-510a-4b31-a16f-587a7c1c97c3 2024-08-25T21:00:08.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T21:00:09.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:00:09.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:00:09.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-08-25T21:00:09.194 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd7 2024-08-25T21:00:09.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T21:00:09.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-08-25T21:00:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T21:00:09.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T21:00:09.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T21:00:09.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 5c54c61f-df56-45b6-a82d-5cc78984bdc4 2024-08-25T21:00:09.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T21:00:09.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T21:00:09.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:09.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T21:00:09.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T21:00:09.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T21:00:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T21:00:09.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary8 2024-08-25T21:00:09.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-08-25T21:00:09.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T21:00:09.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary8 2024-08-25T21:00:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T21:00:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-08-25T21:00:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary8 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary9 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary8 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-08-25T21:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:09.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T21:00:09.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T21:00:09.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T21:00:09.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:09.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-08-25T21:00:09.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T21:00:09.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:09.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T21:00:09.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:00:09.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-08-25T21:00:09.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:09.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:09.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:09.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:09.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-08-25T21:00:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T21:00:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-08-25T21:00:09.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T21:00:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2994 2024-08-25T21:00:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2993 2024-08-25T21:00:09.867 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2992 2024-08-25T21:00:09.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-08-25T21:00:09.881 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:09.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:09.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:09.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:09.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:09.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:09.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:00:09.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:09.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:00:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary9': 2024-08-25T21:00:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T21:00:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T21:00:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T21:00:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3e0f7e5a2981 2024-08-25T21:00:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3e0f7e5a2981 2024-08-25T21:00:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T21:00:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T21:00:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T21:00:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T21:00:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 21:00:09 2024 2024-08-25T21:00:09.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 21:00:09 2024 2024-08-25T21:00:09.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 21:00:09 2024 2024-08-25T21:00:09.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T21:00:09.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T21:00:09.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: 7039707b-c640-4b22-82fe-802cbd6d4b25 2024-08-25T21:00:09.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T21:00:10.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:00:10.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:00:10.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-08-25T21:00:10.188 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd8 2024-08-25T21:00:10.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T21:00:10.189 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-08-25T21:00:10.209 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T21:00:10.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T21:00:10.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T21:00:10.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: c71aec3b-377b-4cc2-90a1-8d6036e799ab 2024-08-25T21:00:10.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T21:00:10.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T21:00:10.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:10.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T21:00:10.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T21:00:10.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T21:00:10.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T21:00:10.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:10.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary9 2024-08-25T21:00:10.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-08-25T21:00:10.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T21:00:10.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary9 2024-08-25T21:00:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T21:00:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-08-25T21:00:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary9 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ MNTPT=test-primary10 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-08-25T21:00:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary9 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n snapshot ']' 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:10.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:10.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:10.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:10.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:10.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local size=128 2024-08-25T21:00:10.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n 10G ']' 2024-08-25T21:00:10.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ size=10G 2024-08-25T21:00:10.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ shift 2024-08-25T21:00:10.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-08-25T21:00:10.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ is_pool_mirror_mode_image mirror 2024-08-25T21:00:10.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:10.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' image = image ']' 2024-08-25T21:00:10.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:00:10.462 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-08-25T21:00:10.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:10.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:10.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:10.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mode=snapshot 2024-08-25T21:00:10.463 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-08-25T21:00:10.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:10.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:10.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:10.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:10.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:10.862 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:11.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:11.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:00:11.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:11.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:11.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:11.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:00:11.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:11.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:11.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:11.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:11.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:11.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:12.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Mirroring enabled 2024-08-25T21:00:12.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2998 2024-08-25T21:00:12.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:12.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-08-25T21:00:12.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2997 2024-08-25T21:00:12.342 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 2996 2024-08-25T21:00:12.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:12.360 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T21:00:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-08-25T21:00:12.583 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:rbd image 'image-primary10': 2024-08-25T21:00:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: size 10 GiB in 2560 objects 2024-08-25T21:00:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: order 22 (4 MiB objects) 2024-08-25T21:00:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: snapshot_count: 1 2024-08-25T21:00:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: id: 3e2210365782 2024-08-25T21:00:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: block_name_prefix: rbd_data.3e2210365782 2024-08-25T21:00:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: format: 2 2024-08-25T21:00:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-08-25T21:00:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: op_features: 2024-08-25T21:00:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: flags: 2024-08-25T21:00:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: create_timestamp: Sun Aug 25 21:00:10 2024 2024-08-25T21:00:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: access_timestamp: Sun Aug 25 21:00:10 2024 2024-08-25T21:00:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: modify_timestamp: Sun Aug 25 21:00:10 2024 2024-08-25T21:00:12.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring state: enabled 2024-08-25T21:00:12.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring mode: snapshot 2024-08-25T21:00:12.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring global id: c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37 2024-08-25T21:00:12.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: mirroring primary: true 2024-08-25T21:00:12.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:00:12.589 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:00:12.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-08-25T21:00:12.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:12.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:00:12.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:12.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:12.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:12.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:00:12.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:12.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:12.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:12.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:12.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:12.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:12.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:12.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:00:12.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:12.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:12.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:12.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:00:13.296 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3002 2024-08-25T21:00:13.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3000 2024-08-25T21:00:13.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3001 2024-08-25T21:00:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEV=/dev/rbd9 2024-08-25T21:00:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ DEVS+=($DEV) 2024-08-25T21:00:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-08-25T21:00:13.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:13.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-08-25T21:00:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:13.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Discarding device blocks: 0/2621440 done 2024-08-25T21:00:13.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-08-25T21:00:13.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Filesystem UUID: 5905f83e-1213-4873-990d-b00bcd328840 2024-08-25T21:00:13.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Superblock backups stored on blocks: 2024-08-25T21:00:13.372 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-08-25T21:00:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Allocating group tables: 0/80 done 2024-08-25T21:00:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing inode tables: 0/80 done 2024-08-25T21:00:13.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Creating journal (16384 blocks): done 2024-08-25T21:00:13.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-08-25T21:00:13.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: 2024-08-25T21:00:13.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mkdir test-primary10 2024-08-25T21:00:13.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-08-25T21:00:13.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ whoami 2024-08-25T21:00:13.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo chown ubuntu test-primary10 2024-08-25T21:00:13.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ SNAP_PIDS+=($!) 2024-08-25T21:00:13.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-08-25T21:00:13.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:13.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:13.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:00:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:00:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ WORKLOAD_PIDS+=($!) 2024-08-25T21:00:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ slow_untar_workload test-primary10 2024-08-25T21:00:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T21:00:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 130489 2024-08-25T21:00:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mountpt=test-primary10 2024-08-25T21:00:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-08-25T21:00:13.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local ret=0 2024-08-25T21:00:13.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-08-25T21:00:14.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3003 2024-08-25T21:00:14.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3005 2024-08-25T21:00:14.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3006 2024-08-25T21:00:14.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3004 2024-08-25T21:00:14.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:14.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:15.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:15.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:00:15.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:15.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:15.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:15.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:00:15.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:15.353 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:00:15.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:15.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:15.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:15.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:00:15.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:15.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:15.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:15.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:15.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:15.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:16.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3009 2024-08-25T21:00:16.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3008 2024-08-25T21:00:16.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3007 2024-08-25T21:00:16.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:16.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:16.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:16.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:16.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:00:16.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:16.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:16.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:16.352 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:00:16.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:16.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:16.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:16.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:16.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:16.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:16.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3011 2024-08-25T21:00:17.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3012 2024-08-25T21:00:17.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3010 2024-08-25T21:00:17.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:17.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:17.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:17.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:17.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:00:17.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:17.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:17.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:17.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:00:17.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:00:17.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:17.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:17.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:17.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:00:17.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:17.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:17.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:17.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:17.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:17.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:17.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:17.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:00:17.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:17.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:17.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:17.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:00:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3015 2024-08-25T21:00:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3016 2024-08-25T21:00:18.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3014 2024-08-25T21:00:18.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:18.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:18.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:18.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3013 2024-08-25T21:00:18.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:19.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:19.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:00:19.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:19.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:19.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:19.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:00:19.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:19.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:19.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:19.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:19.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:19.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:19.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:00:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:00:20.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:20.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:20.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:20.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3019 2024-08-25T21:00:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3017 2024-08-25T21:00:20.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:20.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:00:20.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:20.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:20.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:20.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:00:20.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:20.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3018 2024-08-25T21:00:20.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:20.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:21.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3022 2024-08-25T21:00:21.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:21.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:00:21.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:21.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:21.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:21.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:00:21.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3021 2024-08-25T21:00:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:00:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:00:21.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:21.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:00:21.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:21.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:21.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:21.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:00:21.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:21.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3020 2024-08-25T21:00:21.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:21.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:21.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:21.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:21.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:21.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:21.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:21.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3023 2024-08-25T21:00:21.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3024 2024-08-25T21:00:21.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3025 2024-08-25T21:00:21.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:21.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:22.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:22.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3026 2024-08-25T21:00:22.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:23.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:23.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:00:23.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:23.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:23.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:23.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:00:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:23.120 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:23.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:00:23.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:23.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:23.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:00:23.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3029 2024-08-25T21:00:23.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3027 2024-08-25T21:00:23.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:24.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:24.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:00:24.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:24.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:24.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:24.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:00:24.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:24.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:24.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:24.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:24.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:24.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:24.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:24.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:24.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:24.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:24.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:24.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:24.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:00:24.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:24.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:24.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:24.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:00:24.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3030 2024-08-25T21:00:24.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:24.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:00:24.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:00:25.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:25.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:00:25.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:25.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:25.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:25.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:00:25.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:25.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3028 2024-08-25T21:00:25.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:25.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:25.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:25.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:25.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:25.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:25.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:26.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3034 2024-08-25T21:00:26.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3033 2024-08-25T21:00:26.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:26.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:26.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3032 2024-08-25T21:00:26.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3031 2024-08-25T21:00:26.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:26.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:00:26.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:26.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:26.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:26.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:00:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:00:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:00:26.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:27.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3035 2024-08-25T21:00:27.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:27.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3037 2024-08-25T21:00:27.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:28.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:28.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:00:28.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:28.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:28.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:28.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:00:28.916 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3036 2024-08-25T21:00:28.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:28.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:28.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:28.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:28.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:28.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:28.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:29.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:29.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:00:29.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:29.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:29.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:29.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:00:29.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:29.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:00:29.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:29.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:29.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:29.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:00:29.914 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3038 2024-08-25T21:00:29.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:29.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:29.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:29.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:29.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:29.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:29.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:29.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:29.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:29.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3041 2024-08-25T21:00:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3040 2024-08-25T21:00:30.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:30.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:30.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3039 2024-08-25T21:00:30.968 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:30.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:00:30.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:30.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:30.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:30.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:00:30.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:30.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3044 2024-08-25T21:00:31.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:31.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:00:31.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:31.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:31.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:31.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:00:31.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3043 2024-08-25T21:00:31.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:31.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:31.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3042 2024-08-25T21:00:31.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:31.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3045 2024-08-25T21:00:31.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:00:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:00:32.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3047 2024-08-25T21:00:33.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:00:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:00:33.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:33.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:33.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:33.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:33.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:33.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:33.933 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3046 2024-08-25T21:00:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:00:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:33.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:33.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:00:33.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:34.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:34.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:34.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:34.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:34.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:34.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:34.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3048 2024-08-25T21:00:34.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:34.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:00:34.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:34.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:34.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:34.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:00:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:00:34.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:34.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:34.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:34.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:00:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3051 2024-08-25T21:00:35.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:35.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3049 2024-08-25T21:00:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3050 2024-08-25T21:00:36.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:36.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:36.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3054 2024-08-25T21:00:36.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3052 2024-08-25T21:00:36.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3053 2024-08-25T21:00:36.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:36.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:36.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:36.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:00:36.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:36.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:36.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:36.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:00:37.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:37.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:00:37.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:37.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:37.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:37.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:00:37.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3055 2024-08-25T21:00:38.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:38.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3057 2024-08-25T21:00:38.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:38.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:38.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:00:38.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:38.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:38.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:38.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:00:38.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3056 2024-08-25T21:00:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:39.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:39.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:00:39.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:39.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:39.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:39.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:00:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:00:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:00:39.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:39.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:39.949 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:39.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:39.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:39.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:39.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:39.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:39.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:39.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:39.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:39.950 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:39.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3058 2024-08-25T21:00:40.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:40.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3060 2024-08-25T21:00:40.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:40.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3061 2024-08-25T21:00:40.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:40.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3059 2024-08-25T21:00:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:00:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:00:41.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:00:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:00:41.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3063 2024-08-25T21:00:41.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3064 2024-08-25T21:00:41.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:41.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:41.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3062 2024-08-25T21:00:41.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:41.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3065 2024-08-25T21:00:41.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:42.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:42.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:42.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:42.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:42.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:42.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:42.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3066 2024-08-25T21:00:42.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:43.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3067 2024-08-25T21:00:43.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:43.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:00:43.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:00:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:00:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:00:43.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:43.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:43.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:43.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:43.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:43.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:00:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:44.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:00:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:44.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:44.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:44.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:44.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:44.081 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:44.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:44.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:44.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:44.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:44.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:44.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3068 2024-08-25T21:00:44.201 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:44.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:44.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:00:44.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:44.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:44.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:44.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:00:44.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:44.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:00:44.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:44.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:44.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:44.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:00:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3071 2024-08-25T21:00:45.212 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:45.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:45.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:00:45.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:45.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:45.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:45.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:00:45.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3070 2024-08-25T21:00:45.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3069 2024-08-25T21:00:46.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:46.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:46.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:46.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:46.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:46.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:46.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:46.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3074 2024-08-25T21:00:46.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:46.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3072 2024-08-25T21:00:46.934 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3073 2024-08-25T21:00:46.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3075 2024-08-25T21:00:46.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:46.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:47.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:47.203 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:47.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:00:47.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:47.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:47.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:47.204 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:00:48.036 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:00:48.035+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adfa48c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T21:00:48.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:48.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:00:48.215 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:48.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:48.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:48.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:00:48.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3077 2024-08-25T21:00:49.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3076 2024-08-25T21:00:49.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:49.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:49.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:49.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:49.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:49.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:49.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:49.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:49.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:00:49.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:49.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:49.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:49.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:00:49.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:49.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:49.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:00:49.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:49.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:49.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:49.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:00:49.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:49.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:49.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:49.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:49.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:49.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:50.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:50.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:00:50.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:50.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:50.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:50.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:00:50.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3078 2024-08-25T21:00:50.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:50.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3080 2024-08-25T21:00:50.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:50.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3081 2024-08-25T21:00:50.180 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:51.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3079 2024-08-25T21:00:51.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:51.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:00:51.089+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae3a86000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T21:00:51.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3084 2024-08-25T21:00:51.121 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3083 2024-08-25T21:00:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:51.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:51.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3082 2024-08-25T21:00:51.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3085 2024-08-25T21:00:52.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:52.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:52.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:52.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:52.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:52.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:52.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:52.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:52.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:00:52.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:52.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:52.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:52.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:00:53.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:53.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:00:53.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:53.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:53.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:53.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:00:53.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3087 2024-08-25T21:00:53.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3086 2024-08-25T21:00:53.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:53.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:53.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:53.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:00:53.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:53.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:53.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:53.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:00:53.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:53.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:53.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:53.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:53.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:53.183 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:54.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:54.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:00:54.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:54.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:54.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:54.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:00:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3088 2024-08-25T21:00:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:54.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:54.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:54.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:00:54.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:54.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:54.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:54.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:00:55.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:55.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:00:55.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:55.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:55.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:00:55.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:00:55.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3091 2024-08-25T21:00:55.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:56.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:56.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:00:56.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:56.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:56.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:00:56.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:00:56.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:56.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:00:56.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:56.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:56.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:00:56.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:00:56.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3089 2024-08-25T21:00:56.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3090 2024-08-25T21:00:56.361 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:56.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3094 2024-08-25T21:00:56.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:56.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3092 2024-08-25T21:00:56.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3093 2024-08-25T21:00:56.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:56.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3095 2024-08-25T21:00:57.002 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:57.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3096 2024-08-25T21:00:57.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:57.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:57.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:00:57.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:57.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:57.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:00:57.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:00:57.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3097 2024-08-25T21:00:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:58.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:58.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:00:58.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:58.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:58.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:00:58.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:00:58.945 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3098 2024-08-25T21:00:58.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:00:59.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:59.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:00:59.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:59.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:59.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:00:59.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:00:59.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:59.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:00:59.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:59.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:59.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:00:59.366 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:00:59.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:59.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:00:59.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:59.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:59.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:00:59.423 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:00:59.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:59.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:00:59.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:59.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:59.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:00:59.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:00:59.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:00:59.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:00:59.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:00:59.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:00:59.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:00:59.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:00:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3099 2024-08-25T21:01:00.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:00.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:01:00.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:00.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:00.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:01:00.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:01:00.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3100 2024-08-25T21:01:00.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3101 2024-08-25T21:01:00.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:00.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:01:00.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:00.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:00.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:01:00.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:01:00.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3102 2024-08-25T21:01:00.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:00.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:00.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:00.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:00.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:01:00.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:00.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:00.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:01:00.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:01:01.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3103 2024-08-25T21:01:01.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3105 2024-08-25T21:01:01.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:01.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:01.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:01.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:01:01.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:01.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:01.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:01:01.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:01:02.915 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3106 2024-08-25T21:01:02.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3104 2024-08-25T21:01:02.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:03.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:03.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:03.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:01:03.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:03.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:03.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:01:03.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:01:03.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:03.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:01:03.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:03.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:03.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:01:03.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:01:03.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:03.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:01:03.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:03.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:03.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:01:03.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:01:03.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:03.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:01:03.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:03.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:03.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:01:03.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:01:03.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3107 2024-08-25T21:01:03.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:04.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:04.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:01:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:01:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:01:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:01:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:01:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:01:04.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3108 2024-08-25T21:01:05.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:05.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3111 2024-08-25T21:01:05.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3110 2024-08-25T21:01:05.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3109 2024-08-25T21:01:05.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:05.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:06.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:06.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:01:06.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:06.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:06.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:01:06.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:01:06.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:06.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:01:06.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:06.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:06.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:01:06.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:01:06.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:06.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3113 2024-08-25T21:01:06.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3112 2024-08-25T21:01:06.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3114 2024-08-25T21:01:06.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:06.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:06.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:06.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:06.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:01:06.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:06.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:06.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:01:06.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:01:06.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3115 2024-08-25T21:01:06.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:07.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3117 2024-08-25T21:01:07.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:01:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:01:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:01:08.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3116 2024-08-25T21:01:08.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:01:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:01:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:01:08.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3118 2024-08-25T21:01:08.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:01:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:01:08.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:01:09.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:09.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:01:09.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:09.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:09.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:01:09.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:01:09.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:09.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:01:09.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:09.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:09.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:01:09.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:01:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:01:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:01:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:01:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:01:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:01:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:01:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3121 2024-08-25T21:01:09.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3120 2024-08-25T21:01:09.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3119 2024-08-25T21:01:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:09.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:09.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:01:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:01:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:01:10.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3123 2024-08-25T21:01:10.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3122 2024-08-25T21:01:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3124 2024-08-25T21:01:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:01:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:01:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:01:10.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:10.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:11.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3125 2024-08-25T21:01:11.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:11.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:11.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:01:11.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:11.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:11.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:01:11.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:01:11.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:11.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:01:11.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:11.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:11.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:01:11.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:01:12.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:12.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:01:12.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:12.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:12.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:01:12.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:01:12.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:12.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:01:12.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:12.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:12.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:01:12.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:01:13.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:13.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:01:13.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:13.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:13.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:01:13.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:01:13.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3127 2024-08-25T21:01:13.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:13.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3126 2024-08-25T21:01:13.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:13.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:13.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:01:13.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:13.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:13.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:01:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:01:14.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:14.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:01:14.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:14.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:14.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:01:14.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:01:14.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:14.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:01:14.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:14.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:14.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:01:14.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:01:14.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:14.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:01:14.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:14.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:14.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:01:14.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:01:14.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3128 2024-08-25T21:01:15.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:15.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3129 2024-08-25T21:01:15.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3131 2024-08-25T21:01:15.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3130 2024-08-25T21:01:16.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:16.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:01:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:01:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:01:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:16.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:16.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3132 2024-08-25T21:01:16.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3134 2024-08-25T21:01:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3133 2024-08-25T21:01:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:16.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:16.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:01:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:01:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:01:17.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3135 2024-08-25T21:01:18.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:18.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3137 2024-08-25T21:01:18.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:18.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:01:18.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:18.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:18.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:01:18.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:01:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:18.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3136 2024-08-25T21:01:19.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3138 2024-08-25T21:01:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:01:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:19.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:01:19.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:01:19.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:19.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:01:19.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:19.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:19.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:01:19.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:01:19.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:19.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:19.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:01:19.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:19.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:19.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:01:19.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:01:19.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:19.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:01:19.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:19.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:19.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:01:19.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:01:19.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:19.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:01:19.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:19.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:19.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:01:19.965 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:01:19.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:19.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:01:19.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:19.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:19.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:01:19.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:01:20.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3140 2024-08-25T21:01:20.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3141 2024-08-25T21:01:20.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3139 2024-08-25T21:01:20.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:20.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:20.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:21.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:21.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:01:21.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:21.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:21.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:01:21.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:01:21.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3143 2024-08-25T21:01:21.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3144 2024-08-25T21:01:21.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3142 2024-08-25T21:01:21.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:21.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:01:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:01:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:01:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:21.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3145 2024-08-25T21:01:21.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:01:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:22.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:22.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:01:22.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:01:22.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:22.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:01:22.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:22.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:22.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:01:22.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:01:23.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:23.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:01:23.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:23.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:23.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:01:23.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:01:23.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:23.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:01:23.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:23.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:23.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:01:23.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:01:23.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3147 2024-08-25T21:01:23.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:23.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:01:23.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:23.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:23.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:01:23.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:01:23.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:24.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3146 2024-08-25T21:01:24.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:24.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:01:24.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:24.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:24.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:01:24.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:01:24.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:24.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:01:24.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:24.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:24.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:01:24.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:01:24.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:01:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:01:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:01:24.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:24.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:01:24.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:24.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:24.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:01:24.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:01:25.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3148 2024-08-25T21:01:25.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:26.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3151 2024-08-25T21:01:26.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3150 2024-08-25T21:01:26.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3149 2024-08-25T21:01:26.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:26.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:01:26.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:26.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:26.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:01:26.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:01:26.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:26.073 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:26.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:26.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3154 2024-08-25T21:01:26.921 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3153 2024-08-25T21:01:26.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3152 2024-08-25T21:01:26.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:26.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:26.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:27.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:27.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:01:27.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:27.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:27.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:01:27.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:01:28.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3155 2024-08-25T21:01:28.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:28.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3157 2024-08-25T21:01:28.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:28.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:28.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:01:28.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:28.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:28.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:01:28.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:01:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3156 2024-08-25T21:01:29.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:29.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3158 2024-08-25T21:01:29.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:29.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:01:29.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:29.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:29.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:01:29.075 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:01:29.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:29.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:01:29.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:29.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:29.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:01:29.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:01:29.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:29.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:29.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:01:29.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:29.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:29.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:01:29.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:01:29.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:29.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:01:29.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:29.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:29.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:01:29.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:01:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:01:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:01:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:01:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:01:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:29.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:01:29.961 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:01:30.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3161 2024-08-25T21:01:30.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3159 2024-08-25T21:01:30.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3160 2024-08-25T21:01:30.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:30.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:30.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:31.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:31.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:01:31.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:31.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:31.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:01:31.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:01:31.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3164 2024-08-25T21:01:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3163 2024-08-25T21:01:31.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3162 2024-08-25T21:01:31.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:31.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:31.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:31.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:31.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:01:31.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:31.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:31.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:01:31.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:01:31.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3165 2024-08-25T21:01:31.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:01:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:01:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:01:32.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:32.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:01:32.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:32.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:32.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:01:32.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:01:32.923 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3166 2024-08-25T21:01:32.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3167 2024-08-25T21:01:32.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:33.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:33.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:01:33.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:33.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:33.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:01:33.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:01:33.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:33.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:01:33.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:33.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:33.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:01:33.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:01:33.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:01:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:01:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:01:33.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3168 2024-08-25T21:01:33.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:34.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:34.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:01:34.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:34.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:34.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:01:34.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:01:34.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:34.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:01:34.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:34.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:34.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:01:34.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:01:34.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:34.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:01:34.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:34.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:01:34.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:01:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3170 2024-08-25T21:01:34.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:34.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:01:34.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:34.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:34.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:01:34.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:01:34.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3169 2024-08-25T21:01:34.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3171 2024-08-25T21:01:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:35.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:35.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:35.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:01:35.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:35.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:35.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:01:35.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:01:35.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:35.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:01:35.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:35.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:35.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:01:35.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:01:36.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3174 2024-08-25T21:01:36.924 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3173 2024-08-25T21:01:36.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3172 2024-08-25T21:01:36.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:36.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:01:36.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:36.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:36.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:01:36.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:01:36.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:36.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:36.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3175 2024-08-25T21:01:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:37.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:37.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:01:37.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:37.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:37.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:01:37.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:01:38.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:38.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:01:38.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:38.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:38.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:01:38.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:01:38.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:38.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:01:38.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:38.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:38.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:01:38.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:01:38.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3177 2024-08-25T21:01:38.962 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3176 2024-08-25T21:01:38.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:38.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3178 2024-08-25T21:01:38.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:39.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:39.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:39.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:01:39.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:39.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:39.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:01:39.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:01:39.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3181 2024-08-25T21:01:39.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:39.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:01:39.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:39.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:39.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:01:39.989 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:01:40.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:40.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:01:40.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:40.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:40.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:01:40.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:01:40.016 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:01:40.014+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf913400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T21:01:40.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:40.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:40.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:01:40.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:40.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:40.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:01:40.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:01:40.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3179 2024-08-25T21:01:40.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3180 2024-08-25T21:01:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:41.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3184 2024-08-25T21:01:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3183 2024-08-25T21:01:41.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3182 2024-08-25T21:01:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:41.939 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3185 2024-08-25T21:01:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:41.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:41.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:01:41.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:41.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:41.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:01:41.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:01:41.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:41.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:01:41.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:41.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:41.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:01:41.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:01:42.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:42.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:01:42.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:42.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:42.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:01:42.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:01:42.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3187 2024-08-25T21:01:42.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3186 2024-08-25T21:01:43.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:43.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:43.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:43.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:01:43.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:43.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:43.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:01:43.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:01:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:01:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:01:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:01:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:01:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:43.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:01:43.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:01:44.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3188 2024-08-25T21:01:44.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:44.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:01:44.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:44.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:44.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:01:44.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:01:44.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:44.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:44.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:01:44.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:44.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:44.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:01:44.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:01:44.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:44.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:01:44.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:44.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:44.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:01:44.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:01:44.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:44.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:01:44.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:44.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:44.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:01:44.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:01:44.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3189 2024-08-25T21:01:45.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:45.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3191 2024-08-25T21:01:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3190 2024-08-25T21:01:45.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:45.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:46.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:46.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:01:46.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:46.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:46.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:01:46.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:01:46.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:46.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:01:46.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:46.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:46.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:01:46.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:01:46.918 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3192 2024-08-25T21:01:46.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3194 2024-08-25T21:01:46.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3193 2024-08-25T21:01:46.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:46.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:47.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:47.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:01:47.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:47.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:47.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:01:47.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:01:47.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3195 2024-08-25T21:01:48.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:48.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:48.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:01:48.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:48.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:48.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:01:48.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:01:48.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:48.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:01:48.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:48.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:48.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:01:48.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:01:48.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:48.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:01:48.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:48.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:48.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:01:48.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:01:48.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3196 2024-08-25T21:01:48.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3197 2024-08-25T21:01:49.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:49.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:49.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3200 2024-08-25T21:01:49.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:49.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:01:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:01:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:01:49.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:49.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:01:49.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:49.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:49.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:01:49.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:01:49.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:49.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:01:49.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:49.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:49.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:01:49.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:01:49.982 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3198 2024-08-25T21:01:50.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:50.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3199 2024-08-25T21:01:50.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3201 2024-08-25T21:01:51.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:51.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:01:51.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:51.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:51.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:01:51.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:01:51.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:51.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:51.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3204 2024-08-25T21:01:51.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3202 2024-08-25T21:01:51.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:51.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:51.066 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3203 2024-08-25T21:01:51.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3205 2024-08-25T21:01:51.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:52.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:52.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:01:52.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:52.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:52.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:01:52.009 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:01:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:01:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:01:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:01:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:01:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:01:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:01:53.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:53.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:01:53.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:53.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:53.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:01:53.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:01:53.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3208 2024-08-25T21:01:53.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3206 2024-08-25T21:01:53.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:53.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:53.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3207 2024-08-25T21:01:53.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:54.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3209 2024-08-25T21:01:54.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:54.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:54.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:01:54.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:54.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:54.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:01:54.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:01:54.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:54.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:01:54.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:54.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:54.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:01:54.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:01:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:01:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:01:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:01:54.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:54.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:01:54.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:54.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:54.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:01:54.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:01:54.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:54.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:01:54.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:54.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:54.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:01:54.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:01:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:01:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:01:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:01:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3211 2024-08-25T21:01:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3210 2024-08-25T21:01:55.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:55.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:56.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:56.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:01:56.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:56.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:56.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:01:56.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:01:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:01:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:01:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:01:56.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:56.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:01:56.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:56.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:56.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:01:56.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:01:56.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3214 2024-08-25T21:01:56.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3213 2024-08-25T21:01:56.938 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3212 2024-08-25T21:01:56.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:56.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:56.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:57.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3217 2024-08-25T21:01:57.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:57.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:01:57.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:57.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:57.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:01:57.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:01:57.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:58.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3215 2024-08-25T21:01:58.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:58.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:58.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:01:58.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:58.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:58.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:01:58.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:01:58.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:58.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:01:58.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:58.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:58.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:01:58.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:01:59.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3218 2024-08-25T21:01:59.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3216 2024-08-25T21:01:59.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:59.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:01:59.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:59.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:01:59.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:59.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:59.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:01:59.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:01:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:01:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:01:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:01:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:01:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:01:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:01:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:01:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:01:59.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:02:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3219 2024-08-25T21:02:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:02:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:02:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:02:00.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:01.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3221 2024-08-25T21:02:01.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3220 2024-08-25T21:02:01.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:01.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:01.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:01.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:02:01.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:01.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:01.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:02:01.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:02:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3223 2024-08-25T21:02:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3224 2024-08-25T21:02:01.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:01.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3222 2024-08-25T21:02:01.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:01.115 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:01.953 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3226 2024-08-25T21:02:01.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:02.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:02.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:02:02.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:02.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:02.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:02:02.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:02:02.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:02.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:02:02.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:02.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:02.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:02:02.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:02:02.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3225 2024-08-25T21:02:02.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:02.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3227 2024-08-25T21:02:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3228 2024-08-25T21:02:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:03.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:03.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:02:03.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:03.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:03.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:02:03.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:02:04.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3229 2024-08-25T21:02:04.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:04.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:04.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:02:04.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:04.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:04.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:02:04.060 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:02:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:02:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:02:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:02:04.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:04.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:02:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:02:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:02:04.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:04.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:02:04.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:04.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:04.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:02:04.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:02:04.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:04.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:02:04.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:04.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:04.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:02:04.118 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:02:04.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:04.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:02:04.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:04.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:04.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:02:04.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:02:05.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3231 2024-08-25T21:02:05.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3230 2024-08-25T21:02:05.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:05.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:05.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:05.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:02:05.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:05.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:05.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:02:05.944 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:02:05.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:05.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:02:05.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:05.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:05.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:02:05.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:02:05.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:05.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:02:05.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:05.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:05.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:02:05.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:02:06.920 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3232 2024-08-25T21:02:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3234 2024-08-25T21:02:06.926 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3233 2024-08-25T21:02:06.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:06.966 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:06.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:06.984 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:02:06.983+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555ae36c8000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T21:02:07.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:07.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:02:07.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:07.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:07.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:02:07.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:02:08.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3235 2024-08-25T21:02:08.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:08.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:02:08.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:08.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:08.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:02:08.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:02:08.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:02:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:02:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:02:09.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3238 2024-08-25T21:02:09.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3237 2024-08-25T21:02:09.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:09.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:09.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:09.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:02:09.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:09.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:02:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:02:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:02:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:02:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:02:09.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:09.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:02:09.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:09.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:09.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:02:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:02:10.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3239 2024-08-25T21:02:10.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3240 2024-08-25T21:02:11.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3236 2024-08-25T21:02:11.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3241 2024-08-25T21:02:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:02:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:02:11.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:02:11.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:11.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:11.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3244 2024-08-25T21:02:11.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3242 2024-08-25T21:02:11.149 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3243 2024-08-25T21:02:11.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:11.159 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:11.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:11.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3245 2024-08-25T21:02:11.972 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:12.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:12.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:02:12.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:12.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:12.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:02:12.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:02:12.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:12.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:02:12.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:12.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:12.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:02:12.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:02:13.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3247 2024-08-25T21:02:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3246 2024-08-25T21:02:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:13.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:13.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:13.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:02:13.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:13.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:13.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:02:13.117 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:02:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3248 2024-08-25T21:02:14.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:14.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:02:14.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:14.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:14.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:02:14.100 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:02:14.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:14.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:14.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:02:14.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:14.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:14.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:02:14.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:02:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:02:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:02:14.113 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:02:14.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:14.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:02:14.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:14.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:14.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:02:14.153 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:02:14.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:14.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:02:14.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:14.162 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:14.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:02:14.163 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:02:14.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:14.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:02:14.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:14.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:14.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:02:14.164 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:02:14.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:14.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:02:14.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:14.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:14.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:02:14.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:02:15.123 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3251 2024-08-25T21:02:15.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3250 2024-08-25T21:02:15.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:15.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3249 2024-08-25T21:02:15.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:15.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:16.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:16.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:02:16.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:16.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:16.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:02:16.088 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:02:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:16.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:02:16.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:16.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:16.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:02:16.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:02:16.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3255 2024-08-25T21:02:16.157 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:16.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3253 2024-08-25T21:02:16.929 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3254 2024-08-25T21:02:16.943 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3252 2024-08-25T21:02:16.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:16.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:16.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:17.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:17.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:02:17.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:17.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:17.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:02:17.105 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:02:17.966 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:02:17.965+0000 7f8837a00640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x555adf511400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-08-25T21:02:18.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:18.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:02:18.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:18.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:18.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:02:18.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:02:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:02:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:02:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:02:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:02:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:02:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:02:18.925 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3256 2024-08-25T21:02:18.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3257 2024-08-25T21:02:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3258 2024-08-25T21:02:18.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:18.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:19.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:19.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:19.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:02:19.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:19.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:19.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:02:19.160 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:02:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3260 2024-08-25T21:02:19.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:19.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:02:19.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:19.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:19.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:02:19.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:02:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:02:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:02:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:02:19.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3259 2024-08-25T21:02:19.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3261 2024-08-25T21:02:19.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:19.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:02:19.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:19.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:19.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:02:19.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:02:20.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:20.013 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:20.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3262 2024-08-25T21:02:20.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:20.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:20.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3263 2024-08-25T21:02:20.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3264 2024-08-25T21:02:20.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3265 2024-08-25T21:02:20.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:20.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:20.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:21.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:21.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:02:21.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:21.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:21.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:02:21.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:02:21.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:21.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-08-25T21:02:21.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:21.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:21.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:02:21.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-08-25T21:02:22.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:22.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:02:22.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:22.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:22.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:02:22.004 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:02:22.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3266 2024-08-25T21:02:22.963 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3267 2024-08-25T21:02:22.973 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:22.974 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:23.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:02:23.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:23.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:23.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:02:23.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:02:23.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:23.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-08-25T21:02:23.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:23.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:23.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:02:23.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-08-25T21:02:23.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:02:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:02:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:02:23.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:23.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-08-25T21:02:23.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:23.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:23.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:02:23.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-08-25T21:02:23.969 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3268 2024-08-25T21:02:23.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:23.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:23.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:02:23.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-08-25T21:02:23.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:23.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:23.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:23.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:23.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:02:23.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:02:23.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:02:23.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-08-25T21:02:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-08-25T21:02:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:02:23.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-08-25T21:02:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3272 2024-08-25T21:02:23.999 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:24.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:24.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3270 2024-08-25T21:02:25.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3271 2024-08-25T21:02:25.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3269 2024-08-25T21:02:25.024 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:25.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:25.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:25.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:02:25.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:25.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:25.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:02:25.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:02:25.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T21:02:25.977 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 130621 2024-08-25T21:02:26.919 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3275 2024-08-25T21:02:26.936 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3274 2024-08-25T21:02:26.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3273 2024-08-25T21:02:26.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:26.980 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:26.987 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:27.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:27.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-08-25T21:02:27.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:27.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:27.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:02:27.003 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-08-25T21:02:27.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3276 2024-08-25T21:02:28.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:28.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:28.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:02:28.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:28.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:28.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:02:28.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:02:28.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:28.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-08-25T21:02:28.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:28.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:28.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:02:28.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-08-25T21:02:28.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T21:02:28.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 130753 2024-08-25T21:02:28.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3277 2024-08-25T21:02:29.015 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:29.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T21:02:29.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 130885 2024-08-25T21:02:29.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T21:02:29.984 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 131037 2024-08-25T21:02:29.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T21:02:29.985 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 131190 2024-08-25T21:02:29.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:29.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-08-25T21:02:29.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:29.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:29.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:02:29.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-08-25T21:02:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3279 2024-08-25T21:02:30.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3278 2024-08-25T21:02:30.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:30.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:31.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3280 2024-08-25T21:02:31.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:31.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-08-25T21:02:31.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:31.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:31.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:02:31.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-08-25T21:02:31.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:32.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T21:02:32.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 131364 2024-08-25T21:02:33.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for i in {1..30} 2024-08-25T21:02:33.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-08-25T21:02:33.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:02:33.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:02:33.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:02:33.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-08-25T21:02:33.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T21:02:33.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 131537 2024-08-25T21:02:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3281 2024-08-25T21:02:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:34.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T21:02:34.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 131711 2024-08-25T21:02:35.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Snapshot ID: 3282 2024-08-25T21:02:35.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 3 2024-08-25T21:02:36.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-08-25T21:02:36.997 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 131988 2024-08-25T21:02:38.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T21:02:38.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T21:02:38.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 130490 2024-08-25T21:05:01.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T21:05:01.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T21:05:01.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T21:05:01.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 130622 2024-08-25T21:05:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T21:05:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T21:05:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T21:05:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 130754 2024-08-25T21:05:03.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T21:05:03.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T21:05:03.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T21:05:03.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 130886 2024-08-25T21:05:04.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T21:05:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T21:05:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T21:05:04.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 131038 2024-08-25T21:05:05.750 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T21:05:05.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T21:05:05.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T21:05:05.751 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 131191 2024-08-25T21:05:06.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T21:05:06.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T21:05:06.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T21:05:06.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 131365 2024-08-25T21:05:08.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T21:05:08.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T21:05:08.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T21:05:08.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 131538 2024-08-25T21:05:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T21:05:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T21:05:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T21:05:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 131712 2024-08-25T21:05:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T21:05:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T21:05:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-08-25T21:05:12.461 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 131989 2024-08-25T21:05:13.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ret=124 2024-08-25T21:05:13.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( ret != 124 )) 2024-08-25T21:05:13.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T21:05:13.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:05:13.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T21:05:13.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:05:13.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 1 2024-08-25T21:05:13.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd0 2024-08-25T21:05:13.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary1 2024-08-25T21:05:13.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary1 2024-08-25T21:05:13.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T21:05:13.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary1 2024-08-25T21:05:13.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T21:05:13.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:05:13.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 2 2024-08-25T21:05:13.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T21:05:13.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd1 2024-08-25T21:05:13.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary2 2024-08-25T21:05:13.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:05:13.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary2 2024-08-25T21:05:13.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T21:05:13.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary2 2024-08-25T21:05:13.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 3 2024-08-25T21:05:13.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd2 2024-08-25T21:05:13.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary3 2024-08-25T21:05:13.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary3 2024-08-25T21:05:13.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T21:05:13.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary3 2024-08-25T21:05:13.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T21:05:13.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:05:13.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 4 2024-08-25T21:05:13.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd3 2024-08-25T21:05:13.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary4 2024-08-25T21:05:13.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary4 2024-08-25T21:05:13.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T21:05:13.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary4 2024-08-25T21:05:13.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T21:05:13.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:05:13.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 5 2024-08-25T21:05:13.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd4 2024-08-25T21:05:13.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary5 2024-08-25T21:05:13.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary5 2024-08-25T21:05:13.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T21:05:13.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary5 2024-08-25T21:05:13.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T21:05:13.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:05:13.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 6 2024-08-25T21:05:13.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd5 2024-08-25T21:05:13.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary6 2024-08-25T21:05:13.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary6 2024-08-25T21:05:13.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T21:05:13.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary6 2024-08-25T21:05:13.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T21:05:13.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:05:13.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 7 2024-08-25T21:05:13.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd6 2024-08-25T21:05:13.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary7 2024-08-25T21:05:13.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary7 2024-08-25T21:05:13.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T21:05:13.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary7 2024-08-25T21:05:13.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T21:05:13.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:05:13.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 8 2024-08-25T21:05:13.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T21:05:13.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:05:13.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ COMPARE_PIDS+=($!) 2024-08-25T21:05:13.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T21:05:13.855 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 137827 2024-08-25T21:05:13.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd7 2024-08-25T21:05:13.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary8 2024-08-25T21:05:13.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary8 2024-08-25T21:05:13.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T21:05:13.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary8 2024-08-25T21:05:13.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 9 2024-08-25T21:05:13.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ compare_demoted_promoted_image 10 2024-08-25T21:05:13.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd9 2024-08-25T21:05:13.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary10 2024-08-25T21:05:13.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary10 2024-08-25T21:05:13.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T21:05:13.856 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary10 2024-08-25T21:05:13.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local dev=/dev/rbd8 2024-08-25T21:05:13.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local img=image-primary9 2024-08-25T21:05:13.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local mntpt=test-primary9 2024-08-25T21:05:13.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local demote_md5 promote_md5 2024-08-25T21:05:13.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo umount test-primary9 2024-08-25T21:05:14.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:05:14.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T21:05:14.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:05:14.481 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T21:05:14.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:05:14.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T21:05:14.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T21:05:14.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:05:14.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T21:05:14.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:05:14.721 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T21:05:14.723 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:05:14.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T21:05:14.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T21:05:14.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:05:14.768 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:05:14.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T21:05:14.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:05:14.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T21:05:14.885 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:06:05.400 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=7e2f5e2807ef5cd0ffcdccacb8e8883d 2024-08-25T21:06:05.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-08-25T21:06:05.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary9 2024-08-25T21:06:05.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:05.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:05.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:06:05.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-08-25T21:06:06.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T21:06:06.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-08-25T21:06:06.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:06.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:06.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:06:06.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:06.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:06.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:06.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:06.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=da3862a61ddbbf2969347a58bd3269fb 2024-08-25T21:06:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-08-25T21:06:06.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary1 2024-08-25T21:06:06.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:06.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:06.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:06:06.783 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-08-25T21:06:07.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T21:06:07.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:06:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:07.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:07.114 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T21:06:07.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T21:06:07.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7039707b-c640-4b22-82fe-802cbd6d4b25 2024-08-25T21:06:07.459 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:07.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:07.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:07.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T21:06:09.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=6959afc74e66e3145c041852f2437721 2024-08-25T21:06:09.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-08-25T21:06:09.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T21:06:09.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-08-25T21:06:09.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:09.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:09.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:06:09.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:09.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:09.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:09.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:09.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:09.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=1a2554aece182166d4a50d24ab050bef 2024-08-25T21:06:09.168 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-08-25T21:06:09.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary3 2024-08-25T21:06:09.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:09.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:09.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:06:09.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-08-25T21:06:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T21:06:09.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:09.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:09.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:06:09.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:09.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:09.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:09.484 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:09.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:09.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T21:06:09.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary6 2024-08-25T21:06:09.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:09.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:09.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:06:09.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-08-25T21:06:09.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T21:06:09.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7039707b-c640-4b22-82fe-802cbd6d4b25 2024-08-25T21:06:09.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:09.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:09.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:09.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:09.635 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:09.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:09.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:09.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T21:06:10.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T21:06:10.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T21:06:10.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:10.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:10.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:06:10.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:10.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:10.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:10.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:10.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:10.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T21:06:10.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=451137d310f871de321aff01241b59b7 2024-08-25T21:06:10.116 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-08-25T21:06:10.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-08-25T21:06:10.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:10.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:10.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:06:10.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:10.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:10.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:10.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:10.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=1a1e435909aa4789cf72b4a85fb16e00 2024-08-25T21:06:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-08-25T21:06:10.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T21:06:10.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: aadbded2-2750-407c-895a-1bb2e92d5a94 2024-08-25T21:06:10.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:10.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:10.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:10.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:10.249 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:10.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:10.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:10.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T21:06:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary7 2024-08-25T21:06:10.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:10.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:10.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:06:10.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-08-25T21:06:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary2 2024-08-25T21:06:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:06:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-08-25T21:06:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=776f528893fd2103f2e07bf758d2eaec 2024-08-25T21:06:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-08-25T21:06:10.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=64e2c02241c9d1212ffdaeb292020440 2024-08-25T21:06:10.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-08-25T21:06:10.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary4 2024-08-25T21:06:10.614 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:10.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:10.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:06:10.615 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-08-25T21:06:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary10 2024-08-25T21:06:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:06:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-08-25T21:06:10.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=bda87f042817f778fc1d05483c3295cf 2024-08-25T21:06:10.978 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-08-25T21:06:11.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T21:06:11.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T21:06:11.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-08-25T21:06:11.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:11.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:11.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:06:11.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:11.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:11.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:11.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:11.102 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:11.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T21:06:11.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:11.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:11.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:06:11.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:11.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:11.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-08-25T21:06:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:06:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:11.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:11.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T21:06:11.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary8 2024-08-25T21:06:11.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:11.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:11.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:06:11.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-08-25T21:06:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_md5=9d0538d2c4b6e28a69727d61366ce1fd 2024-08-25T21:06:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-08-25T21:06:11.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T21:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eac9715-6778-491b-b159-ca14ad42f707 2024-08-25T21:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:11.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T21:06:11.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ demote_image cluster1 mirror image-primary5 2024-08-25T21:06:11.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:11.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:11.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:06:11.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-08-25T21:06:12.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T21:06:12.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T21:06:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T21:06:12.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-08-25T21:06:12.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:12.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:12.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:06:12.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:12.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:12.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:12.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:12.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:12.086 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-08-25T21:06:12.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:12.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:12.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:06:12.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:12.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:12.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:12.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:12.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:12.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-08-25T21:06:12.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:12.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:12.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:06:12.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:12.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:12.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:12.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:12.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:12.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T21:06:12.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:12.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:12.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:06:12.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:12.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:12.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:12.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:12.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:12.110 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T21:06:12.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T21:06:12.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:12.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:12.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:06:12.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:12.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:12.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:12.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:12.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:12.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T21:06:12.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T21:06:12.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:12.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:12.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:06:12.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:12.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:12.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:12.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:12.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:12.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T21:06:12.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T21:06:12.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 547a99b7-e2b8-4de2-9d6e-c0c889ebf727 2024-08-25T21:06:12.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:12.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:12.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:12.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:12.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:12.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:12.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:12.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T21:06:12.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T21:06:12.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a6d3b19e-9e6d-435d-9a57-070b64a2d592 2024-08-25T21:06:12.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:12.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:12.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:12.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:12.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T21:06:12.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T21:06:12.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: aadbded2-2750-407c-895a-1bb2e92d5a94 2024-08-25T21:06:12.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:12.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:12.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:12.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:12.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:12.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:12.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:12.518 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T21:06:13.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T21:06:13.089 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-08-25T21:06:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:06:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:13.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T21:06:13.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:13.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:13.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:06:13.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:13.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:13.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:13.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:13.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T21:06:13.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:13.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:13.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:06:13.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T21:06:13.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:13.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T21:06:13.101 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T21:06:13.103 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:06:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:13.104 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:13.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:13.107 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T21:06:13.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T21:06:13.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9de1696e-ed4d-41f7-8c4e-02953f84c7dc 2024-08-25T21:06:13.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:13.280 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:13.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:13.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:13.281 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:13.284 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:13.285 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:13.286 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T21:06:13.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T21:06:13.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37 2024-08-25T21:06:13.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:13.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:13.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:13.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:13.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:13.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T21:06:13.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9ffbd30-cb37-47e1-9712-592083b16894 2024-08-25T21:06:13.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:13.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:13.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:13.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:13.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:13.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:13.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:13.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T21:06:13.308 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:13.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:13.309 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T21:06:13.362 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T21:06:13.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:13.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:13.363 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:06:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:13.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:13.371 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T21:06:13.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T21:06:13.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eac9715-6778-491b-b159-ca14ad42f707 2024-08-25T21:06:13.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:13.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:13.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:13.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:13.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:13.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:13.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T21:06:13.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T21:06:13.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:13.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:13.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:06:13.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:13.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:13.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:13.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:13.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:13.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T21:06:13.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T21:06:13.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7039707b-c640-4b22-82fe-802cbd6d4b25 2024-08-25T21:06:13.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:13.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:13.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:13.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:13.714 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:13.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:13.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:13.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T21:06:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:14.093 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:06:14.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:14.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:14.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:14.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:14.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:14.094 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T21:06:14.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T21:06:14.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: abb0fd4b-510a-4b31-a16f-587a7c1c97c3 2024-08-25T21:06:14.170 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:14.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:14.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:14.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:14.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:14.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:14.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:14.173 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T21:06:14.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T21:06:14.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:14.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:14.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:06:14.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:14.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:14.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:14.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:14.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:14.314 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T21:06:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T21:06:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 547a99b7-e2b8-4de2-9d6e-c0c889ebf727 2024-08-25T21:06:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:14.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:14.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T21:06:14.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T21:06:14.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:14.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:14.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:06:14.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:14.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:14.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:14.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:14.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:14.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T21:06:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T21:06:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a6d3b19e-9e6d-435d-9a57-070b64a2d592 2024-08-25T21:06:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:14.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:14.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:14.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T21:06:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T21:06:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:06:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T21:06:15.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T21:06:15.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:15.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:15.310 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:06:15.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:15.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:15.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:15.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:15.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:15.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T21:06:15.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T21:06:15.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:15.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:15.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:06:15.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:15.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:15.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:15.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:15.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:15.313 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T21:06:15.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T21:06:15.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9de1696e-ed4d-41f7-8c4e-02953f84c7dc 2024-08-25T21:06:15.364 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:15.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:15.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:15.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:15.365 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:15.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:15.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:15.367 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T21:06:15.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T21:06:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9ffbd30-cb37-47e1-9712-592083b16894 2024-08-25T21:06:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T21:06:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37 2024-08-25T21:06:15.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:15.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:15.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:15.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:15.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:15.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:15.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:15.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T21:06:15.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:15.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:15.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T21:06:16.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image demoted to non-primary 2024-08-25T21:06:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-08-25T21:06:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:06:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:16.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:16.175 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T21:06:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:06:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:16.177 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T21:06:16.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T21:06:16.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: abb0fd4b-510a-4b31-a16f-587a7c1c97c3 2024-08-25T21:06:16.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:16.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:16.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:16.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:16.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:16.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:16.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:16.253 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T21:06:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T21:06:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:06:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T21:06:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T21:06:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: aadbded2-2750-407c-895a-1bb2e92d5a94 2024-08-25T21:06:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:16.600 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:16.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:16.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:16.602 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:17.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T21:06:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:06:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T21:06:17.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T21:06:17.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ad28100c-6a8b-4e3f-9d4a-e05c1aff7120 2024-08-25T21:06:17.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:17.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:17.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:17.166 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:17.167 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:17.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:17.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:17.169 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 2 2024-08-25T21:06:17.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T21:06:17.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:17.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:17.496 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:06:17.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:17.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:17.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:17.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:17.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:17.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T21:06:17.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T21:06:17.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eac9715-6778-491b-b159-ca14ad42f707 2024-08-25T21:06:17.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:17.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:17.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:17.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:17.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:17.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:17.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:17.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:18.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T21:06:18.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:06:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T21:06:18.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T21:06:18.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:18.474 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:18.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:06:18.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:18.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:18.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:18.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:18.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:18.475 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T21:06:18.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T21:06:18.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 547a99b7-e2b8-4de2-9d6e-c0c889ebf727 2024-08-25T21:06:18.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:18.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:18.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:18.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:18.488 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:18.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:18.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:18.491 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:18.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T21:06:18.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a6d3b19e-9e6d-435d-9a57-070b64a2d592 2024-08-25T21:06:18.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:18.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:18.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:18.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:18.555 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:19.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T21:06:19.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:19.171 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:19.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:06:19.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:19.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:19.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:19.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:19.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:19.172 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T21:06:19.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T21:06:19.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ad28100c-6a8b-4e3f-9d4a-e05c1aff7120 2024-08-25T21:06:19.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:19.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:19.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:19.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:19.255 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:19.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:19.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:19.258 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 4 2024-08-25T21:06:19.369 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T21:06:19.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:19.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:19.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:06:19.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:19.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:19.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:19.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:19.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:19.370 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T21:06:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T21:06:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T21:06:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:06:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:06:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T21:06:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T21:06:19.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T21:06:19.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9de1696e-ed4d-41f7-8c4e-02953f84c7dc 2024-08-25T21:06:19.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:19.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:19.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:19.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:19.448 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:19.450 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:19.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:19.451 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:19.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T21:06:19.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9ffbd30-cb37-47e1-9712-592083b16894 2024-08-25T21:06:19.465 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:19.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:19.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:19.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:19.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:19.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T21:06:19.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37 2024-08-25T21:06:19.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:19.466 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:19.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:19.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:19.467 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:19.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:19.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:19.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:19.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:19.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:19.469 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T21:06:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:06:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:20.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:20.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T21:06:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T21:06:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: abb0fd4b-510a-4b31-a16f-587a7c1c97c3 2024-08-25T21:06:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:20.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:20.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:20.343 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:21.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T21:06:21.719 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:21.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:21.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:06:21.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:21.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:21.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:21.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:21.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:21.720 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T21:06:21.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T21:06:21.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7039707b-c640-4b22-82fe-802cbd6d4b25 2024-08-25T21:06:21.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:21.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:21.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:21.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:21.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:21.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:21.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:21.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:23.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T21:06:23.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:23.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:23.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:06:23.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:23.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:23.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:23.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:23.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:23.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T21:06:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T21:06:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ad28100c-6a8b-4e3f-9d4a-e05c1aff7120 2024-08-25T21:06:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:23.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:23.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:23.348 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:24.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T21:06:24.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:24.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:24.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:06:24.605 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:24.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:24.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:24.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:24.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:24.606 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T21:06:24.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T21:06:24.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: aadbded2-2750-407c-895a-1bb2e92d5a94 2024-08-25T21:06:24.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:24.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:24.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:24.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:24.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:24.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:24.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:24.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:25.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T21:06:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:06:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T21:06:25.666 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T21:06:25.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eac9715-6778-491b-b159-ca14ad42f707 2024-08-25T21:06:25.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:25.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:25.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:25.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:25.667 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:25.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:25.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:25.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T21:06:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:06:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:26.494 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T21:06:26.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T21:06:26.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:26.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:26.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:06:26.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:26.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:26.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:26.560 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:26.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:26.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T21:06:26.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T21:06:26.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 547a99b7-e2b8-4de2-9d6e-c0c889ebf727 2024-08-25T21:06:26.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:26.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:26.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:26.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:26.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:26.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:26.581 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:26.582 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:26.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T21:06:26.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a6d3b19e-9e6d-435d-9a57-070b64a2d592 2024-08-25T21:06:26.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:26.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:26.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:26.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:26.642 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:27.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T21:06:27.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:27.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:27.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:06:27.453 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:27.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:27.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:27.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:27.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:27.454 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T21:06:27.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T21:06:27.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T21:06:27.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:27.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:27.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:27.471 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:06:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:06:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:27.472 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:27.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T21:06:27.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:27.473 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T21:06:27.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T21:06:27.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9de1696e-ed4d-41f7-8c4e-02953f84c7dc 2024-08-25T21:06:27.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:27.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:27.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:27.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:27.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:27.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:27.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:27.538 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9ffbd30-cb37-47e1-9712-592083b16894 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+stopped 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: local image is primary 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:05:58' 2024-08-25T21:06:27.552 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:27.553 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:27.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 1 2024-08-25T21:06:27.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:27.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:27.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:27.554 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 8 2024-08-25T21:06:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-08-25T21:06:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:06:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:28.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:28.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:28.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:28.347 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T21:06:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T21:06:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: abb0fd4b-510a-4b31-a16f-587a7c1c97c3 2024-08-25T21:06:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:28.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:28.428 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:28.430 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:28.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:28.431 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:28.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:28.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:28.434 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:28.436 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:28.437 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:28.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:28.441 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:28.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:28.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:28.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:28.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:06:28.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:28.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:28.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:28.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:28.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:28.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:28.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:28.551 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T21:06:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:28.587 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T21:06:28.590 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:28.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:28.591 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:28.593 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-08-25T21:06:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:06:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:28.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:28.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:28.595 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:29.596 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-08-25T21:06:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:06:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:29.598 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-08-25T21:06:29.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T21:06:29.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: abb0fd4b-510a-4b31-a16f-587a7c1c97c3 2024-08-25T21:06:29.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:29.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:29.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:29.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:29.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:29.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:29.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:29.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:29.677 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:29.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:29.678 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:29.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:29.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:29.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:29.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:29.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:29.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:29.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:29.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:29.688 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:06:29.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:29.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:29.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:29.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:29.689 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:29.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:29.793 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:29.794 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T21:06:29.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T21:06:29.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:29.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:29.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:29.805 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:29.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:29.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:29.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary8 2024-08-25T21:06:29.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:29.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:29.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:06:29.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T21:06:29.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-08-25T21:06:29.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-08-25T21:06:29.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:29.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:29.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:06:29.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:29.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:29.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:29.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:29.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:29.811 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T21:06:29.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T21:06:29.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7039707b-c640-4b22-82fe-802cbd6d4b25 2024-08-25T21:06:29.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:29.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:29.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:29.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:29.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:29.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:29.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:29.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:29.897 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:29.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:29.898 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:29.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:29.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:29.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:29.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:29.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:29.909 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:29.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:29.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:06:29.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:29.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:29.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:29.910 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:29.911 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:29.912 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T21:06:29.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:06:29.927 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:06:29.928 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-08-25T21:06:30.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:30.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:30.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T21:06:30.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:30.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T21:06:30.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:30.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:30.034 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-08-25T21:06:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:06:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:30.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:30.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:30.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:30.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd0 2024-08-25T21:06:30.062 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd0 2024-08-25T21:06:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:06:31.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-08-25T21:06:31.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:31.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:31.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:06:31.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:31.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:31.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:31.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:31.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:31.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-08-25T21:06:31.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T21:06:31.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7039707b-c640-4b22-82fe-802cbd6d4b25 2024-08-25T21:06:31.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:31.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:31.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:31.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:31.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:31.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:31.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:31.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:31.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:31.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:31.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:31.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:31.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:31.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:31.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:31.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:06:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:31.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:31.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:31.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T21:06:31.256 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:31.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T21:06:31.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:31.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:31.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:31.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:31.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:31.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:31.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary9 2024-08-25T21:06:31.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:31.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:31.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:06:31.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T21:06:31.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-08-25T21:06:31.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-08-25T21:06:31.350 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:06:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T21:06:31.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T21:06:31.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ad28100c-6a8b-4e3f-9d4a-e05c1aff7120 2024-08-25T21:06:31.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:31.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:31.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:31.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:31.427 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:31.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:31.429 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:31.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:31.433 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:31.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:31.435 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:31.439 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:31.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:31.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:31.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:31.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:06:31.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:31.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:31.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:31.443 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:31.444 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:31.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:31.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:31.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T21:06:31.561 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:31.562 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T21:06:31.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:31.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:31.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-08-25T21:06:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:06:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:31.570 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:31.917 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T21:06:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:06:31.930 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:06:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-08-25T21:06:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd1 2024-08-25T21:06:32.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd1 2024-08-25T21:06:32.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:06:32.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-08-25T21:06:32.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:32.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:32.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:06:32.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:32.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:32.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:32.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:32.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:32.573 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-08-25T21:06:32.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T21:06:32.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ad28100c-6a8b-4e3f-9d4a-e05c1aff7120 2024-08-25T21:06:32.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:32.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:32.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:32.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:32.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:32.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:32.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:32.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:32.651 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:32.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:32.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:32.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:32.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:32.657 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:06:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:32.661 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:32.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:32.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:32.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-08-25T21:06:32.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:32.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:32.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:06:32.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:32.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:32.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:32.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:32.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:32.694 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T21:06:32.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:32.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:32.792 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T21:06:32.797 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:32.798 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T21:06:32.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T21:06:32.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: aadbded2-2750-407c-895a-1bb2e92d5a94 2024-08-25T21:06:32.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:32.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:32.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:32.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:32.801 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:32.802 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:32.803 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:32.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:32.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:32.804 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:32.806 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:32.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:32.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:32.807 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:32.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:32.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:32.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary5 2024-08-25T21:06:32.808 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:32.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:32.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:06:32.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T21:06:32.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:32.809 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-08-25T21:06:32.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:32.810 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:32.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:32.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:32.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:32.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:32.818 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:06:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:32.932 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T21:06:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:32.941 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T21:06:32.946 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:32.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T21:06:32.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:06:32.947 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:06:32.948 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-08-25T21:06:32.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:32.951 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:32.952 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:32.954 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:32.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:32.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:32.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-08-25T21:06:32.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:32.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:32.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:06:32.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:32.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:32.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:32.955 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:33.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd2 2024-08-25T21:06:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd2 2024-08-25T21:06:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:06:33.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-08-25T21:06:33.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:33.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:33.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:06:33.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:33.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:33.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:33.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:33.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:33.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T21:06:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T21:06:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eac9715-6778-491b-b159-ca14ad42f707 2024-08-25T21:06:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:33.752 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:33.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:33.755 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:33.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:33.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:33.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:33.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:33.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:33.762 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:33.766 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:33.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:33.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:33.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:06:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T21:06:33.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:33.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T21:06:33.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:33.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:33.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:33.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:33.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-08-25T21:06:33.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:33.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:33.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:06:33.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:33.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:33.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:33.904 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:33.905 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:33.957 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-08-25T21:06:33.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:33.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:33.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:06:33.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:33.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:33.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:33.958 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:33.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:33.959 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-08-25T21:06:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T21:06:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: aadbded2-2750-407c-895a-1bb2e92d5a94 2024-08-25T21:06:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:34.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:34.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:34.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:34.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:34.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:34.040 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:34.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:34.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:34.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:34.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:34.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:06:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T21:06:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T21:06:34.192 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:34.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:34.193 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:34.195 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:34.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:34.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:34.196 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary1 2024-08-25T21:06:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:06:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T21:06:34.197 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-08-25T21:06:34.584 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-08-25T21:06:34.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:34.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:34.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:06:34.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:34.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:34.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:34.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:34.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:34.585 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T21:06:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-08-25T21:06:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:06:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:34.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T21:06:34.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T21:06:34.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 547a99b7-e2b8-4de2-9d6e-c0c889ebf727 2024-08-25T21:06:34.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:34.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:34.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:34.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:34.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:34.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:34.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:34.673 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:34.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:34.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:34.675 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:34.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:34.679 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:34.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:34.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:34.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:34.692 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:06:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:34.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T21:06:34.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a6d3b19e-9e6d-435d-9a57-070b64a2d592 2024-08-25T21:06:34.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:34.738 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:34.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:34.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:34.739 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:34.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:34.744 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:34.748 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:34.749 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:34.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:34.753 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:34.754 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:34.758 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:34.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:34.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:34.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:34.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:06:34.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:34.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:34.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:34.759 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:34.760 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T21:06:34.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:34.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T21:06:34.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:34.838 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:34.839 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:34.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:34.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:34.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:34.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-08-25T21:06:34.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:34.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:34.844 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:06:34.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:34.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:34.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:34.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:34.845 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:34.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:34.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:34.878 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T21:06:34.883 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:34.884 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T21:06:34.888 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:34.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:34.889 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:34.891 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-08-25T21:06:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:34.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:34.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:06:34.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:34.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:34.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:34.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:34.893 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:34.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-08-25T21:06:34.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:34.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:34.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:06:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:34.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-08-25T21:06:34.964 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T21:06:34.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:06:34.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:06:34.981 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-08-25T21:06:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T21:06:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eac9715-6778-491b-b159-ca14ad42f707 2024-08-25T21:06:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:35.007 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:35.011 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:35.012 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:35.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:35.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:35.022 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:35.023 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:35.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:35.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:35.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:35.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:35.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:35.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:06:35.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:35.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:35.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:35.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:35.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:35.182 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd3 2024-08-25T21:06:35.186 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd3 2024-08-25T21:06:35.187 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:06:35.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:35.207 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T21:06:35.213 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:35.214 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T21:06:35.221 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:35.222 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:35.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:35.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:35.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:35.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary3 2024-08-25T21:06:35.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:35.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:06:35.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T21:06:35.226 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-08-25T21:06:35.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-08-25T21:06:35.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:35.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:06:35.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:35.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:35.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:35.541 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:35.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:35.542 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T21:06:35.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-08-25T21:06:35.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-08-25T21:06:35.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:35.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:35.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:06:35.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:35.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.556 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:35.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:06:35.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:35.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:35.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:35.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:35.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:35.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:35.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:35.557 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T21:06:35.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:35.558 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T21:06:35.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T21:06:35.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9de1696e-ed4d-41f7-8c4e-02953f84c7dc 2024-08-25T21:06:35.643 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:35.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:35.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:35.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:35.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:35.647 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:35.648 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:35.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:35.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:35.654 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:35.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:35.655 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T21:06:35.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37 2024-08-25T21:06:35.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:35.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:35.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:35.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:35.656 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:35.658 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:35.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:35.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:35.659 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:35.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:35.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:35.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:35.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:35.663 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:35.664 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:35.665 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:35.668 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:35.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:35.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:35.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:35.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:06:35.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:35.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:35.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:35.669 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:35.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:35.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:35.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:35.672 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:35.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T21:06:35.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9ffbd30-cb37-47e1-9712-592083b16894 2024-08-25T21:06:35.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:35.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:35.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:35.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:35.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:35.674 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:35.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:35.676 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:35.680 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:06:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:35.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:35.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:35.682 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:35.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:35.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:35.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:35.686 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:35.690 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:35.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:35.693 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:35.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:35.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:35.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:06:35.699 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:06:35.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:35.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:35.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:35.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:35.701 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:35.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:35.820 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:35.821 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T21:06:35.830 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:35.831 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T21:06:35.840 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:06:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:35.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-08-25T21:06:35.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:35.851 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:35.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T21:06:35.858 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:35.859 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T21:06:35.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:35.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:35.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:35.868 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:35.869 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-08-25T21:06:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:06:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:35.870 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:35.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41904272mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:35.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:35.882 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T21:06:35.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:35.887 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T21:06:35.894 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-08-25T21:06:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:06:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:35.895 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:35.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:35.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:35.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:35.896 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-08-25T21:06:35.899 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-08-25T21:06:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:35.900 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:35.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:06:35.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:35.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:35.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:35.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:06:35.901 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:06:35.971 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T21:06:35.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:06:35.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:06:35.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-08-25T21:06:35.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T21:06:35.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 547a99b7-e2b8-4de2-9d6e-c0c889ebf727 2024-08-25T21:06:35.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:35.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:35.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:35.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:35.996 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:36.000 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:36.001 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:36.006 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:36.008 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:36.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:36.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:36.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:36.014 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:36.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:36.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:36.021 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:36.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T21:06:36.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a6d3b19e-9e6d-435d-9a57-070b64a2d592 2024-08-25T21:06:36.026 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:36.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:36.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:36.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:36.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:36.027 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:06:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:36.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:36.030 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:36.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:36.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:36.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:36.038 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:36.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:36.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:36.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:36.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:36.049 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:06:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:36.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:36.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:36.260 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd4 2024-08-25T21:06:36.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd4 2024-08-25T21:06:36.268 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:06:36.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:36.273 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:36.274 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T21:06:36.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:36.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T21:06:36.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:36.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:36.299 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:36.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:36.302 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:36.303 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary6 2024-08-25T21:06:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:06:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T21:06:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-08-25T21:06:36.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T21:06:36.311 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:36.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T21:06:36.325 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary4 2024-08-25T21:06:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:06:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T21:06:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-08-25T21:06:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-08-25T21:06:36.852 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:36.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:36.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:06:36.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:36.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:36.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:36.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:36.853 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:36.854 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-08-25T21:06:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-08-25T21:06:36.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:36.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:36.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:06:36.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:36.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:36.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:36.873 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:36.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:36.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-08-25T21:06:36.906 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-08-25T21:06:36.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:36.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:36.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:06:36.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:36.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:06:36.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:06:36.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:36.907 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:36.908 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-08-25T21:06:37.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T21:06:37.035 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T21:06:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T21:06:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9de1696e-ed4d-41f7-8c4e-02953f84c7dc 2024-08-25T21:06:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:37.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:37.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:37.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:37.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:37.048 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:37.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:37.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:37.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:37.061 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:37.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:37.065 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:06:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:06:37.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T21:06:37.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37 2024-08-25T21:06:37.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:37.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:37.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:37.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:37.069 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-08-25T21:06:37.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:37.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:37.074 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:37.076 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:37.082 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:37.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:06:37.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:06:37.083 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:37.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T21:06:37.084 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9ffbd30-cb37-47e1-9712-592083b16894 2024-08-25T21:06:37.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+unknown 2024-08-25T21:06:37.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: remote image is not primary 2024-08-25T21:06:37.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:06:37.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:06:28' 2024-08-25T21:06:37.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+unknown' 2024-08-25T21:06:37.085 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-08-25T21:06:37.087 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:37.092 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:37.095 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:37.096 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+unknown 2024-08-25T21:06:37.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:37.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:37.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:37.097 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:37.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:37.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:06:37.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:37.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:37.098 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:37.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:37.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:37.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:37.099 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:06:37.106 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:37.108 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:37.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: remote image is not primary 2024-08-25T21:06:37.109 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:06:37.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:37.111 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:06:37.112 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:37.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:37.119 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:06:37.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:37.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:06:37.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:37.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:37.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:06:37.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:37.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:06:37.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:37.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:37.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:37.127 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:06:37.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:37.129 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:37.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:37.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-08-25 21:06:28' 2024-08-25T21:06:37.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:37.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:37.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:06:37.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+unknown 2024-08-25T21:06:37.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:06:28' 2024-08-25T21:06:37.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:06:37.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:06:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-08-25T21:06:37.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd5 2024-08-25T21:06:37.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:06:37.520 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd5 2024-08-25T21:06:37.536 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd6 2024-08-25T21:06:37.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:37.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:37.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T21:06:37.547 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd6 2024-08-25T21:06:37.550 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:06:37.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:37.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T21:06:37.604 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:37.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:37.608 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:37.618 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:37.619 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:37.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:37.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary7 2024-08-25T21:06:37.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:37.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:37.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:06:37.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T21:06:37.620 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-08-25T21:06:37.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:37.634 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T21:06:37.662 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:37.670 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T21:06:37.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041974236mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+unknownremote image is not primary41974236mirror.0smithi1212024-08-25 21:06:28' 2024-08-25T21:06:37.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:06:37.683 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:37.685 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T21:06:37.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:37.687 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:37.691 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:37.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:37.697 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:37.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary2 2024-08-25T21:06:37.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:37.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:37.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:06:37.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T21:06:37.698 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-08-25T21:06:37.700 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:06:28' 2024-08-25T21:06:37.702 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T21:06:37.717 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+unknown 2024-08-25T21:06:37.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+unknown 2024-08-25T21:06:37.722 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+unknown 2024-08-25T21:06:37.724 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+unknown 2024-08-25T21:06:37.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:37.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:06:37.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_image cluster2 mirror image-primary10 2024-08-25T21:06:37.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:06:37.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:06:37.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:06:37.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local force= 2024-08-25T21:06:37.726 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-08-25T21:06:38.025 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T21:06:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T21:06:38.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:06:38.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:06:38.071 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-08-25T21:06:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:06:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:06:38.091 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-08-25T21:06:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd7 2024-08-25T21:06:38.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd7 2024-08-25T21:06:38.485 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:06:38.493 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd8 2024-08-25T21:06:38.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:06:38.499 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd8 2024-08-25T21:06:39.018 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:Image promoted to primary 2024-08-25T21:06:39.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \n\b\d ]] 2024-08-25T21:06:39.064 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ krbd == \k\r\b\d ]] 2024-08-25T21:06:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-08-25T21:06:39.482 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ dev=/dev/rbd9 2024-08-25T21:06:39.490 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ awk '{print $1}' 2024-08-25T21:06:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sudo md5sum /dev/rbd9 2024-08-25T21:07:19.813 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=bda87f042817f778fc1d05483c3295cf 2024-08-25T21:07:19.814 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-08-25T21:07:20.090 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ bda87f042817f778fc1d05483c3295cf != \b\d\a\8\7\f\0\4\2\8\1\7\f\7\7\8\f\c\1\d\0\5\4\8\3\c\3\2\9\5\c\f ]] 2024-08-25T21:07:22.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=7e2f5e2807ef5cd0ffcdccacb8e8883d 2024-08-25T21:07:22.770 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-08-25T21:07:23.039 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 7e2f5e2807ef5cd0ffcdccacb8e8883d != \7\e\2\f\5\e\2\8\0\7\e\f\5\c\d\0\f\f\c\d\c\c\a\c\b\8\e\8\8\8\3\d ]] 2024-08-25T21:07:23.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=9d0538d2c4b6e28a69727d61366ce1fd 2024-08-25T21:07:23.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-08-25T21:07:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 9d0538d2c4b6e28a69727d61366ce1fd != \9\d\0\5\3\8\d\2\c\4\b\6\e\2\8\a\6\9\7\2\7\d\6\1\3\6\6\c\e\1\f\d ]] 2024-08-25T21:07:29.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=1a1e435909aa4789cf72b4a85fb16e00 2024-08-25T21:07:29.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-08-25T21:07:29.261 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=6959afc74e66e3145c041852f2437721 2024-08-25T21:07:29.262 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-08-25T21:07:29.335 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 1a1e435909aa4789cf72b4a85fb16e00 != \1\a\1\e\4\3\5\9\0\9\a\a\4\7\8\9\c\f\7\2\b\4\a\8\5\f\b\1\6\e\0\0 ]] 2024-08-25T21:07:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 6959afc74e66e3145c041852f2437721 != \6\9\5\9\a\f\c\7\4\e\6\6\e\3\1\4\5\c\0\4\1\8\5\2\f\2\4\3\7\7\2\1 ]] 2024-08-25T21:07:29.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=da3862a61ddbbf2969347a58bd3269fb 2024-08-25T21:07:29.715 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-08-25T21:07:29.849 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=1a2554aece182166d4a50d24ab050bef 2024-08-25T21:07:29.850 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-08-25T21:07:29.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ da3862a61ddbbf2969347a58bd3269fb != \d\a\3\8\6\2\a\6\1\d\d\b\b\f\2\9\6\9\3\4\7\a\5\8\b\d\3\2\6\9\f\b ]] 2024-08-25T21:07:29.864 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T21:07:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 137828 2024-08-25T21:07:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T21:07:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 137830 2024-08-25T21:07:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T21:07:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 137831 2024-08-25T21:07:30.028 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 1a2554aece182166d4a50d24ab050bef != \1\a\2\5\5\4\a\e\c\e\1\8\2\1\6\6\d\4\a\5\0\d\2\4\a\b\0\5\0\b\e\f ]] 2024-08-25T21:07:30.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=776f528893fd2103f2e07bf758d2eaec 2024-08-25T21:07:30.254 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-08-25T21:07:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 776f528893fd2103f2e07bf758d2eaec != \7\7\6\f\5\2\8\8\9\3\f\d\2\1\0\3\f\2\e\0\7\b\f\7\5\8\d\2\e\a\e\c ]] 2024-08-25T21:07:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T21:07:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 137834 2024-08-25T21:07:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T21:07:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 137836 2024-08-25T21:07:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T21:07:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 137838 2024-08-25T21:07:30.746 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=64e2c02241c9d1212ffdaeb292020440 2024-08-25T21:07:30.747 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-08-25T21:07:30.874 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 64e2c02241c9d1212ffdaeb292020440 != \6\4\e\2\c\0\2\2\4\1\c\9\d\1\2\1\2\f\f\d\a\e\b\2\9\2\0\2\0\4\4\0 ]] 2024-08-25T21:07:31.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ promote_md5=451137d310f871de321aff01241b59b7 2024-08-25T21:07:31.042 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-08-25T21:07:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ [[ 451137d310f871de321aff01241b59b7 != \4\5\1\1\3\7\d\3\1\0\f\8\7\1\d\e\3\2\1\a\f\f\0\1\2\4\1\b\5\9\b\7 ]] 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 137840 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 137842 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait 137843 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ (( RET != 0 )) 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary1 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:31.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:07:31.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T21:07:31.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-08-25T21:07:31.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:31.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:31.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:07:31.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:31.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:31.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:31.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:31.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:07:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-08-25T21:07:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:32.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:07:32.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:32.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:32.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:32.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:32.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:32.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-08-25T21:07:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary1: 2024-08-25T21:07:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: aadbded2-2750-407c-895a-1bb2e92d5a94 2024-08-25T21:07:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T21:07:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3249561.6,"bytes_per_snapshot":64991232.0,"last_snapshot_bytes":64991232,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619994,"remote_snapshot_timestamp":1724619994,"replay_state":"idle"} 2024-08-25T21:07:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:07:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:07:28' 2024-08-25T21:07:32.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T21:07:32.231 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T21:07:32.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:07:32.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3249561.6,"bytes_per_snapshot":64991232.0,"last_snapshot_bytes":64991232,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619994,"remote_snapshot_timestamp":1724619994,"replay_state":"idle"} 2024-08-25T21:07:32.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:07:32.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:32.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:07:32.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:32.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:07:32.240 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:07:32.241 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:07:32.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:07:32.245 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-08-25 21:07:28' 2024-08-25T21:07:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:07:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:07:28' 2024-08-25T21:07:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:32.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:07:32.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING1041904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary1aadbded2-2750-407c-895a-1bb2e92d5a94up+replayingreplaying, {"bytes_per_second":3249561.6,"bytes_per_snapshot":64991232.0,"last_snapshot_bytes":64991232,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619994,"remote_snapshot_timestamp":1724619994,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+replayingreplaying, {"bytes_per_second":3339059.2,"bytes_per_snapshot":66781184.0,"last_snapshot_bytes":66781184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619998,"remote_snapshot_timestamp":1724619998,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+replayingreplaying, {"bytes_per_second":3247718.4,"bytes_per_snapshot":64954368.0,"last_snapshot_bytes":64954368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+replayingreplaying, {"bytes_per_second":3173990.4,"bytes_per_snapshot":63479808.0,"last_snapshot_bytes":63479808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619995,"remote_snapshot_timestamp":1724619995,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+replayingreplaying, {"bytes_per_second":3105792.0,"bytes_per_snapshot":62115840.0,"last_snapshot_bytes":62115840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+replayingreplaying, {"bytes_per_second":3426099.2,"bytes_per_snapshot":68521984.0,"last_snapshot_bytes":68521984,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619992,"remote_snapshot_timestamp":1724619992,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+replayingreplaying, {"bytes_per_second":3273523.2,"bytes_per_snapshot":65470464.0,"last_snapshot_bytes":65470464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+replayingreplaying, {"bytes_per_second":3270451.2,"bytes_per_snapshot":65409024.0,"last_snapshot_bytes":65409024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+replayingreplaying, {"bytes_per_second":3188736.0,"bytes_per_snapshot":63774720.0,"last_snapshot_bytes":63774720,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619989,"remote_snapshot_timestamp":1724619989,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+replayingreplaying, {"bytes_per_second":3297075.2,"bytes_per_snapshot":65941504.0,"last_snapshot_bytes":65941504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619991,"remote_snapshot_timestamp":1724619991,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28' 2024-08-25T21:07:32.354 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:07:32.355 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-08-25T21:07:32.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:32.398 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-08-25T21:07:32.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T21:07:32.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:32.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T21:07:32.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:32.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:32.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:32.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary1 2024-08-25T21:07:32.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:07:32.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:32.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:07:32.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-08-25T21:07:32.468 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-08-25T21:07:32.612 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:32.611+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4c3b860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T21:07:32.613 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:32.611+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ae4fe2a00 [3/aadbded2-2750-407c-895a-1bb2e92d5a94] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T21:07:34.210 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T21:07:34.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-08-25T21:07:34.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:34.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:34.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary1 2024-08-25T21:07:34.216 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:34.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T21:07:34.217 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary1 2024-08-25T21:07:34.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:34.271 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary1 2024-08-25T21:07:34.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:07:34.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary2 2024-08-25T21:07:34.289 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:34.290 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:07:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-08-25T21:07:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:07:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:35.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:35.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-08-25T21:07:35.373 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary2: 2024-08-25T21:07:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: b9ffbd30-cb37-47e1-9712-592083b16894 2024-08-25T21:07:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T21:07:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3247718.4,"bytes_per_snapshot":64954368.0,"last_snapshot_bytes":64954368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"} 2024-08-25T21:07:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:07:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:07:28' 2024-08-25T21:07:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T21:07:35.376 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T21:07:35.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:07:35.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3247718.4,"bytes_per_snapshot":64954368.0,"last_snapshot_bytes":64954368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"} 2024-08-25T21:07:35.379 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:07:35.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:35.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:07:35.382 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:35.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:07:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:07:35.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:07:35.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:07:35.390 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-08-25 21:07:28' 2024-08-25T21:07:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:07:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:07:28' 2024-08-25T21:07:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:35.391 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:07:35.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING941904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+replayingreplaying, {"bytes_per_second":3339059.2,"bytes_per_snapshot":66781184.0,"last_snapshot_bytes":66781184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619998,"remote_snapshot_timestamp":1724619998,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary2b9ffbd30-cb37-47e1-9712-592083b16894up+replayingreplaying, {"bytes_per_second":3247718.4,"bytes_per_snapshot":64954368.0,"last_snapshot_bytes":64954368,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+replayingreplaying, {"bytes_per_second":3173990.4,"bytes_per_snapshot":63479808.0,"last_snapshot_bytes":63479808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619995,"remote_snapshot_timestamp":1724619995,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+replayingreplaying, {"bytes_per_second":3105792.0,"bytes_per_snapshot":62115840.0,"last_snapshot_bytes":62115840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+replayingreplaying, {"bytes_per_second":3426099.2,"bytes_per_snapshot":68521984.0,"last_snapshot_bytes":68521984,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619992,"remote_snapshot_timestamp":1724619992,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+replayingreplaying, {"bytes_per_second":3273523.2,"bytes_per_snapshot":65470464.0,"last_snapshot_bytes":65470464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+replayingreplaying, {"bytes_per_second":3270451.2,"bytes_per_snapshot":65409024.0,"last_snapshot_bytes":65409024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+replayingreplaying, {"bytes_per_second":3188736.0,"bytes_per_snapshot":63774720.0,"last_snapshot_bytes":63774720,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619989,"remote_snapshot_timestamp":1724619989,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+replayingreplaying, {"bytes_per_second":3297075.2,"bytes_per_snapshot":65941504.0,"last_snapshot_bytes":65941504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619991,"remote_snapshot_timestamp":1724619991,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28' 2024-08-25T21:07:35.497 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:07:35.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-08-25T21:07:35.501 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:35.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-08-25T21:07:35.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T21:07:35.506 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:35.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T21:07:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary2 2024-08-25T21:07:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:07:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:07:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-08-25T21:07:35.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-08-25T21:07:35.717 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:35.716+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ade8bc680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T21:07:35.717 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:35.716+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ae365fb80 [3/b9ffbd30-cb37-47e1-9712-592083b16894] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T21:07:37.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T21:07:37.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-08-25T21:07:37.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:37.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:37.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary2 2024-08-25T21:07:37.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:37.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T21:07:37.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary2 2024-08-25T21:07:37.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:37.287 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary2 2024-08-25T21:07:37.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:07:37.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary3 2024-08-25T21:07:37.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-08-25T21:07:37.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:37.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:37.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:07:37.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T21:07:37.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-08-25T21:07:37.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:37.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:07:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:07:38.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-08-25T21:07:38.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:38.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:38.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:07:38.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:38.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:38.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:38.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:38.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:38.293 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-08-25T21:07:38.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary3: 2024-08-25T21:07:38.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 5eac9715-6778-491b-b159-ca14ad42f707 2024-08-25T21:07:38.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T21:07:38.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3173990.4,"bytes_per_snapshot":63479808.0,"last_snapshot_bytes":63479808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619995,"remote_snapshot_timestamp":1724619995,"replay_state":"idle"} 2024-08-25T21:07:38.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:07:38.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:07:28' 2024-08-25T21:07:38.375 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T21:07:38.377 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T21:07:38.378 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:07:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3173990.4,"bytes_per_snapshot":63479808.0,"last_snapshot_bytes":63479808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619995,"remote_snapshot_timestamp":1724619995,"replay_state":"idle"} 2024-08-25T21:07:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:07:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:07:38.383 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:38.384 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:07:38.386 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:07:38.387 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:07:38.388 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:07:38.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:38.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-08-25 21:07:28' 2024-08-25T21:07:38.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:38.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:38.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:07:38.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:38.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:07:28' 2024-08-25T21:07:38.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:38.392 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:38.393 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:07:38.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING841904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+replayingreplaying, {"bytes_per_second":3339059.2,"bytes_per_snapshot":66781184.0,"last_snapshot_bytes":66781184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619998,"remote_snapshot_timestamp":1724619998,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary35eac9715-6778-491b-b159-ca14ad42f707up+replayingreplaying, {"bytes_per_second":3173990.4,"bytes_per_snapshot":63479808.0,"last_snapshot_bytes":63479808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619995,"remote_snapshot_timestamp":1724619995,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+replayingreplaying, {"bytes_per_second":3105792.0,"bytes_per_snapshot":62115840.0,"last_snapshot_bytes":62115840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+replayingreplaying, {"bytes_per_second":3426099.2,"bytes_per_snapshot":68521984.0,"last_snapshot_bytes":68521984,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619992,"remote_snapshot_timestamp":1724619992,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+replayingreplaying, {"bytes_per_second":3273523.2,"bytes_per_snapshot":65470464.0,"last_snapshot_bytes":65470464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+replayingreplaying, {"bytes_per_second":3270451.2,"bytes_per_snapshot":65409024.0,"last_snapshot_bytes":65409024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+replayingreplaying, {"bytes_per_second":3188736.0,"bytes_per_snapshot":63774720.0,"last_snapshot_bytes":63774720,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619989,"remote_snapshot_timestamp":1724619989,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+replayingreplaying, {"bytes_per_second":3297075.2,"bytes_per_snapshot":65941504.0,"last_snapshot_bytes":65941504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619991,"remote_snapshot_timestamp":1724619991,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28' 2024-08-25T21:07:38.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:07:38.498 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-08-25T21:07:38.502 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:38.503 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-08-25T21:07:38.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T21:07:38.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:38.507 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T21:07:38.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary3 2024-08-25T21:07:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:07:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:07:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-08-25T21:07:38.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-08-25T21:07:38.723 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:38.722+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3331040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T21:07:38.723 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:38.722+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972c80 [3/5eac9715-6778-491b-b159-ca14ad42f707] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T21:07:40.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T21:07:40.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-08-25T21:07:40.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:40.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:40.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary3 2024-08-25T21:07:40.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:40.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T21:07:40.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary3 2024-08-25T21:07:40.291 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:40.292 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary3 2024-08-25T21:07:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:07:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary4 2024-08-25T21:07:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:07:41.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-08-25T21:07:41.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:41.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:41.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:07:41.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:41.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:41.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:41.297 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:41.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:41.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-08-25T21:07:41.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary4: 2024-08-25T21:07:41.401 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: a6d3b19e-9e6d-435d-9a57-070b64a2d592 2024-08-25T21:07:41.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T21:07:41.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3105792.0,"bytes_per_snapshot":62115840.0,"last_snapshot_bytes":62115840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"} 2024-08-25T21:07:41.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:07:41.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:07:28' 2024-08-25T21:07:41.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T21:07:41.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T21:07:41.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:07:41.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3105792.0,"bytes_per_snapshot":62115840.0,"last_snapshot_bytes":62115840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"} 2024-08-25T21:07:41.407 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:07:41.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:41.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:07:41.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:41.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:07:41.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:07:41.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:07:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:07:41.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:41.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-08-25 21:07:28' 2024-08-25T21:07:41.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:41.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:41.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:07:41.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:41.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:07:28' 2024-08-25T21:07:41.418 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:41.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:41.419 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:07:41.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING741904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+replayingreplaying, {"bytes_per_second":3339059.2,"bytes_per_snapshot":66781184.0,"last_snapshot_bytes":66781184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619998,"remote_snapshot_timestamp":1724619998,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary4a6d3b19e-9e6d-435d-9a57-070b64a2d592up+replayingreplaying, {"bytes_per_second":3105792.0,"bytes_per_snapshot":62115840.0,"last_snapshot_bytes":62115840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+replayingreplaying, {"bytes_per_second":3426099.2,"bytes_per_snapshot":68521984.0,"last_snapshot_bytes":68521984,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619992,"remote_snapshot_timestamp":1724619992,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+replayingreplaying, {"bytes_per_second":3273523.2,"bytes_per_snapshot":65470464.0,"last_snapshot_bytes":65470464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+replayingreplaying, {"bytes_per_second":3270451.2,"bytes_per_snapshot":65409024.0,"last_snapshot_bytes":65409024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+replayingreplaying, {"bytes_per_second":3188736.0,"bytes_per_snapshot":63774720.0,"last_snapshot_bytes":63774720,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619989,"remote_snapshot_timestamp":1724619989,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+replayingreplaying, {"bytes_per_second":3297075.2,"bytes_per_snapshot":65941504.0,"last_snapshot_bytes":65941504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619991,"remote_snapshot_timestamp":1724619991,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28' 2024-08-25T21:07:41.566 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:07:41.567 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-08-25T21:07:41.571 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:41.572 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-08-25T21:07:41.575 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T21:07:41.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:41.576 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T21:07:41.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:41.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:41.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:41.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary4 2024-08-25T21:07:41.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:07:41.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:41.579 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:07:41.580 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-08-25T21:07:41.641 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-08-25T21:07:41.783 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:41.782+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adf23c340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T21:07:41.783 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:41.782+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973b80 [3/a6d3b19e-9e6d-435d-9a57-070b64a2d592] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T21:07:43.244 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T21:07:43.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-08-25T21:07:43.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:43.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:43.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary4 2024-08-25T21:07:43.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:43.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T21:07:43.252 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary4 2024-08-25T21:07:43.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:43.301 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary4 2024-08-25T21:07:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:07:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary5 2024-08-25T21:07:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-08-25T21:07:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:07:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T21:07:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-08-25T21:07:43.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:43.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:43.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:07:43.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:43.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:43.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:43.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:43.305 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:07:44.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-08-25T21:07:44.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:44.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:44.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:07:44.306 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:44.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:44.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:44.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:44.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:44.307 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-08-25T21:07:44.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary5: 2024-08-25T21:07:44.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: ad28100c-6a8b-4e3f-9d4a-e05c1aff7120 2024-08-25T21:07:44.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T21:07:44.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3426099.2,"bytes_per_snapshot":68521984.0,"last_snapshot_bytes":68521984,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619992,"remote_snapshot_timestamp":1724619992,"replay_state":"idle"} 2024-08-25T21:07:44.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:07:44.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:07:28' 2024-08-25T21:07:44.522 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T21:07:44.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T21:07:44.524 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:07:44.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3426099.2,"bytes_per_snapshot":68521984.0,"last_snapshot_bytes":68521984,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619992,"remote_snapshot_timestamp":1724619992,"replay_state":"idle"} 2024-08-25T21:07:44.527 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:07:44.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:44.529 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:07:44.530 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:44.531 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:07:44.533 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:07:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:07:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:07:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-08-25 21:07:28' 2024-08-25T21:07:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:07:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:07:28' 2024-08-25T21:07:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:07:44.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING641904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+replayingreplaying, {"bytes_per_second":3339059.2,"bytes_per_snapshot":66781184.0,"last_snapshot_bytes":66781184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619998,"remote_snapshot_timestamp":1724619998,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary5ad28100c-6a8b-4e3f-9d4a-e05c1aff7120up+replayingreplaying, {"bytes_per_second":3426099.2,"bytes_per_snapshot":68521984.0,"last_snapshot_bytes":68521984,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619992,"remote_snapshot_timestamp":1724619992,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+replayingreplaying, {"bytes_per_second":3273523.2,"bytes_per_snapshot":65470464.0,"last_snapshot_bytes":65470464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+replayingreplaying, {"bytes_per_second":3270451.2,"bytes_per_snapshot":65409024.0,"last_snapshot_bytes":65409024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+replayingreplaying, {"bytes_per_second":3188736.0,"bytes_per_snapshot":63774720.0,"last_snapshot_bytes":63774720,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619989,"remote_snapshot_timestamp":1724619989,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+replayingreplaying, {"bytes_per_second":3297075.2,"bytes_per_snapshot":65941504.0,"last_snapshot_bytes":65941504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619991,"remote_snapshot_timestamp":1724619991,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28' 2024-08-25T21:07:44.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:07:44.640 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-08-25T21:07:44.644 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:44.645 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-08-25T21:07:44.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T21:07:44.649 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:44.650 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T21:07:44.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:44.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:44.652 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:44.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary5 2024-08-25T21:07:44.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:07:44.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:44.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:07:44.653 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-08-25T21:07:44.716 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-08-25T21:07:44.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:44.860+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adfb67520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T21:07:44.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:44.860+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ae365f900 [3/ad28100c-6a8b-4e3f-9d4a-e05c1aff7120] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T21:07:46.259 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T21:07:46.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-08-25T21:07:46.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:46.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:46.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary5 2024-08-25T21:07:46.265 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:46.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T21:07:46.266 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary5 2024-08-25T21:07:46.316 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:46.317 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary5 2024-08-25T21:07:46.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:07:46.319 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary6 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:46.321 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:07:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-08-25T21:07:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:07:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-08-25T21:07:47.396 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary6: 2024-08-25T21:07:47.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 547a99b7-e2b8-4de2-9d6e-c0c889ebf727 2024-08-25T21:07:47.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T21:07:47.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3273523.2,"bytes_per_snapshot":65470464.0,"last_snapshot_bytes":65470464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"} 2024-08-25T21:07:47.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:07:47.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:07:28' 2024-08-25T21:07:47.397 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T21:07:47.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T21:07:47.399 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:07:47.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3273523.2,"bytes_per_snapshot":65470464.0,"last_snapshot_bytes":65470464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"} 2024-08-25T21:07:47.402 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:07:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:07:47.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:47.405 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:07:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:07:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:07:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:07:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-08-25 21:07:28' 2024-08-25T21:07:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:07:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:07:28' 2024-08-25T21:07:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:07:47.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING541904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+replayingreplaying, {"bytes_per_second":3339059.2,"bytes_per_snapshot":66781184.0,"last_snapshot_bytes":66781184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619998,"remote_snapshot_timestamp":1724619998,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary6547a99b7-e2b8-4de2-9d6e-c0c889ebf727up+replayingreplaying, {"bytes_per_second":3273523.2,"bytes_per_snapshot":65470464.0,"last_snapshot_bytes":65470464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619996,"remote_snapshot_timestamp":1724619996,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+replayingreplaying, {"bytes_per_second":3270451.2,"bytes_per_snapshot":65409024.0,"last_snapshot_bytes":65409024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+replayingreplaying, {"bytes_per_second":3188736.0,"bytes_per_snapshot":63774720.0,"last_snapshot_bytes":63774720,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619989,"remote_snapshot_timestamp":1724619989,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+replayingreplaying, {"bytes_per_second":3297075.2,"bytes_per_snapshot":65941504.0,"last_snapshot_bytes":65941504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619991,"remote_snapshot_timestamp":1724619991,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28' 2024-08-25T21:07:47.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:07:47.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-08-25T21:07:47.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:47.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-08-25T21:07:47.512 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T21:07:47.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:47.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T21:07:47.515 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary6 2024-08-25T21:07:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:07:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:07:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-08-25T21:07:47.577 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-08-25T21:07:47.722 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:47.720+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4fcb520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T21:07:47.722 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:47.720+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade972780 [3/547a99b7-e2b8-4de2-9d6e-c0c889ebf727] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T21:07:49.257 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T21:07:49.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-08-25T21:07:49.263 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:49.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:49.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary6 2024-08-25T21:07:49.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:49.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T21:07:49.264 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary6 2024-08-25T21:07:49.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:49.326 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary6 2024-08-25T21:07:49.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:07:49.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary7 2024-08-25T21:07:49.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-08-25T21:07:49.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:49.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:49.330 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:07:49.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T21:07:49.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-08-25T21:07:49.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:49.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:49.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:07:49.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:49.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:49.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:49.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:49.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:07:50.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-08-25T21:07:50.331 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:07:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-08-25T21:07:50.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary7: 2024-08-25T21:07:50.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 9de1696e-ed4d-41f7-8c4e-02953f84c7dc 2024-08-25T21:07:50.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T21:07:50.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3270451.2,"bytes_per_snapshot":65409024.0,"last_snapshot_bytes":65409024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"} 2024-08-25T21:07:50.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:07:50.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:07:28' 2024-08-25T21:07:50.404 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T21:07:50.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T21:07:50.406 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:07:50.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3270451.2,"bytes_per_snapshot":65409024.0,"last_snapshot_bytes":65409024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"} 2024-08-25T21:07:50.409 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:07:50.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:50.410 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:07:50.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:50.412 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:07:50.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:07:50.415 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:07:50.416 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:07:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-08-25 21:07:28' 2024-08-25T21:07:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:07:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:07:28' 2024-08-25T21:07:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:50.421 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:07:50.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING441904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+replayingreplaying, {"bytes_per_second":3339059.2,"bytes_per_snapshot":66781184.0,"last_snapshot_bytes":66781184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619998,"remote_snapshot_timestamp":1724619998,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary79de1696e-ed4d-41f7-8c4e-02953f84c7dcup+replayingreplaying, {"bytes_per_second":3270451.2,"bytes_per_snapshot":65409024.0,"last_snapshot_bytes":65409024,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619997,"remote_snapshot_timestamp":1724619997,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+replayingreplaying, {"bytes_per_second":3188736.0,"bytes_per_snapshot":63774720.0,"last_snapshot_bytes":63774720,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619989,"remote_snapshot_timestamp":1724619989,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+replayingreplaying, {"bytes_per_second":3297075.2,"bytes_per_snapshot":65941504.0,"last_snapshot_bytes":65941504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619991,"remote_snapshot_timestamp":1724619991,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28' 2024-08-25T21:07:50.504 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:07:50.505 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-08-25T21:07:50.508 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:50.509 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-08-25T21:07:50.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T21:07:50.513 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:50.514 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T21:07:50.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:50.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:50.516 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:50.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary7 2024-08-25T21:07:50.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:07:50.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:50.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:07:50.517 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-08-25T21:07:50.578 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-08-25T21:07:50.728 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:50.727+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae3c18ea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T21:07:50.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:50.727+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972500 [3/9de1696e-ed4d-41f7-8c4e-02953f84c7dc] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T21:07:51.983 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T21:07:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-08-25T21:07:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary7 2024-08-25T21:07:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T21:07:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary7 2024-08-25T21:07:52.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:52.041 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary7 2024-08-25T21:07:52.043 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:07:52.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary8 2024-08-25T21:07:52.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-08-25T21:07:52.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:52.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:52.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:07:52.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T21:07:52.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-08-25T21:07:52.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:52.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:52.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:07:52.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:52.044 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:52.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:52.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:52.045 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:07:53.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-08-25T21:07:53.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:53.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:53.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:07:53.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:53.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:53.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:53.046 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:53.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:53.047 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-08-25T21:07:53.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary8: 2024-08-25T21:07:53.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: abb0fd4b-510a-4b31-a16f-587a7c1c97c3 2024-08-25T21:07:53.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T21:07:53.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3188736.0,"bytes_per_snapshot":63774720.0,"last_snapshot_bytes":63774720,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619989,"remote_snapshot_timestamp":1724619989,"replay_state":"idle"} 2024-08-25T21:07:53.125 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:07:53.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:07:28' 2024-08-25T21:07:53.126 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T21:07:53.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T21:07:53.128 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:07:53.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3188736.0,"bytes_per_snapshot":63774720.0,"last_snapshot_bytes":63774720,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619989,"remote_snapshot_timestamp":1724619989,"replay_state":"idle"} 2024-08-25T21:07:53.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:07:53.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:53.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:07:53.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:53.134 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:07:53.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:07:53.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:07:53.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:07:53.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-08-25 21:07:28' 2024-08-25T21:07:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:07:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:07:28' 2024-08-25T21:07:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:53.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:53.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:07:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING341904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+replayingreplaying, {"bytes_per_second":3339059.2,"bytes_per_snapshot":66781184.0,"last_snapshot_bytes":66781184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619998,"remote_snapshot_timestamp":1724619998,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary8abb0fd4b-510a-4b31-a16f-587a7c1c97c3up+replayingreplaying, {"bytes_per_second":3188736.0,"bytes_per_snapshot":63774720.0,"last_snapshot_bytes":63774720,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619989,"remote_snapshot_timestamp":1724619989,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+replayingreplaying, {"bytes_per_second":3297075.2,"bytes_per_snapshot":65941504.0,"last_snapshot_bytes":65941504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619991,"remote_snapshot_timestamp":1724619991,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28' 2024-08-25T21:07:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:07:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-08-25T21:07:53.229 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:53.230 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-08-25T21:07:53.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T21:07:53.234 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:53.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T21:07:53.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:53.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:53.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:53.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary8 2024-08-25T21:07:53.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:07:53.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:53.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:07:53.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-08-25T21:07:53.300 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-08-25T21:07:53.437 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:53.436+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4ed7a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T21:07:53.438 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:53.436+0000 7f8847400640 -1 rbd::mirror::ImageReplayer: 0x555ade973180 [3/abb0fd4b-510a-4b31-a16f-587a7c1c97c3] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T21:07:54.988 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T21:07:54.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-08-25T21:07:54.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:54.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:54.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary8 2024-08-25T21:07:54.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:54.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T21:07:54.995 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary8 2024-08-25T21:07:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:55.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary8 2024-08-25T21:07:55.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:07:55.054 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary9 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:07:56.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-08-25T21:07:56.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:56.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:56.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:07:56.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:56.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:56.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:56.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:56.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:56.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-08-25T21:07:56.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary9: 2024-08-25T21:07:56.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: 7039707b-c640-4b22-82fe-802cbd6d4b25 2024-08-25T21:07:56.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T21:07:56.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":3297075.2,"bytes_per_snapshot":65941504.0,"last_snapshot_bytes":65941504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619991,"remote_snapshot_timestamp":1724619991,"replay_state":"idle"} 2024-08-25T21:07:56.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:07:56.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:07:28' 2024-08-25T21:07:56.138 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T21:07:56.140 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T21:07:56.141 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:07:56.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":3297075.2,"bytes_per_snapshot":65941504.0,"last_snapshot_bytes":65941504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619991,"remote_snapshot_timestamp":1724619991,"replay_state":"idle"} 2024-08-25T21:07:56.144 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:07:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:07:56.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:56.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:07:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:07:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:07:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:07:56.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:56.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-08-25 21:07:28' 2024-08-25T21:07:56.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:56.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:56.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:07:56.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:56.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:07:28' 2024-08-25T21:07:56.155 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:56.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:56.156 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:07:56.237 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING241904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+replayingreplaying, {"bytes_per_second":3339059.2,"bytes_per_snapshot":66781184.0,"last_snapshot_bytes":66781184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619998,"remote_snapshot_timestamp":1724619998,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28image-primary97039707b-c640-4b22-82fe-802cbd6d4b25up+replayingreplaying, {"bytes_per_second":3297075.2,"bytes_per_snapshot":65941504.0,"last_snapshot_bytes":65941504,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619991,"remote_snapshot_timestamp":1724619991,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:28' 2024-08-25T21:07:56.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:07:56.238 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-08-25T21:07:56.242 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:28' 2024-08-25T21:07:56.243 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-08-25T21:07:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T21:07:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T21:07:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary9 2024-08-25T21:07:56.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:07:56.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:56.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:07:56.251 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-08-25T21:07:56.312 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-08-25T21:07:56.459 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:56.457+0000 7f883fc00640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555adf720ea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T21:07:56.459 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:56.457+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ae4fe2000 [3/7039707b-c640-4b22-82fe-802cbd6d4b25] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T21:07:57.986 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T21:07:57.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-08-25T21:07:57.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:57.993 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:57.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary9 2024-08-25T21:07:57.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:57.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T21:07:57.994 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary9 2024-08-25T21:07:58.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:58.053 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary9 2024-08-25T21:07:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for j in {1..10} 2024-08-25T21:07:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ IMG=image-primary10 2024-08-25T21:07:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-08-25T21:07:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:07:58.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' snapshot = journal ']' 2024-08-25T21:07:58.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-08-25T21:07:58.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:58.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:58.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:07:58.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:58.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:58.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:58.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:07:58.057 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ sleep 1 2024-08-25T21:07:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-08-25T21:07:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:07:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local description_pattern= 2024-08-25T21:07:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local service_pattern= 2024-08-25T21:07:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-08-25T21:07:59.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='image-primary10: 2024-08-25T21:07:59.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: global_id: c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37 2024-08-25T21:07:59.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: state: up+replaying 2024-08-25T21:07:59.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":66781184.0,"last_snapshot_bytes":66781184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619998,"remote_snapshot_timestamp":1724619998,"replay_state":"idle"} 2024-08-25T21:07:59.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: service: mirror.0 on smithi121 2024-08-25T21:07:59.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: last_update: 2024-08-25 21:07:58' 2024-08-25T21:07:59.131 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ state: .*up+replaying' 2024-08-25T21:07:59.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: state: up+replaying 2024-08-25T21:07:59.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^ description: .*' 2024-08-25T21:07:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":66781184.0,"last_snapshot_bytes":66781184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619998,"remote_snapshot_timestamp":1724619998,"replay_state":"idle"} 2024-08-25T21:07:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -n '' ']' 2024-08-25T21:07:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep '^up+' 2024-08-25T21:07:59.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:59.139 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep 'service: *mirror..* on ' 2024-08-25T21:07:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout: service: mirror.0 on smithi121 2024-08-25T21:07:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update 2024-08-25T21:07:59.143 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-08-25T21:07:59.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:58' 2024-08-25T21:07:59.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-08-25 21:07:58' 2024-08-25T21:07:59.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:07:59.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:59.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:07:59.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local state_pattern=up+replaying 2024-08-25T21:07:59.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local 'prev_last_update=2024-08-25 21:07:58' 2024-08-25T21:07:59.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local status 2024-08-25T21:07:59.147 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ CEPH_ARGS= 2024-08-25T21:07:59.148 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-08-25T21:07:59.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ status='WARNINGOKWARNING141904272mirror.0smithi12119.0.0-5971-gdb4959e4trueOKimage-primary10c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":66781184.0,"last_snapshot_bytes":66781184,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1724619998,"remote_snapshot_timestamp":1724619998,"replay_state":"idle"}41904272mirror.0smithi1212024-08-25 21:07:58' 2024-08-25T21:07:59.223 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local last_update state 2024-08-25T21:07:59.224 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-08-25T21:07:59.227 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ last_update='2024-08-25 21:07:58' 2024-08-25T21:07:59.228 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-08-25T21:07:59.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ state=up+replaying 2024-08-25T21:07:59.232 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo up+replaying 2024-08-25T21:07:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep up+replaying 2024-08-25T21:07:59.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:up+replaying 2024-08-25T21:07:59.235 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:59.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:07:59.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ remove_image cluster2 mirror image-primary10 2024-08-25T21:07:59.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:07:59.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:07:59.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:07:59.236 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-08-25T21:07:59.298 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-08-25T21:07:59.441 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:59.440+0000 7f883f200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555ae4fca340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-08-25T21:07:59.442 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:07:59.440+0000 7f8847e00640 -1 rbd::mirror::ImageReplayer: 0x555ade972280 [3/c3abc5a5-5aa3-4ccc-ae70-5d86a12f3f37] operator(): start failed: (85) Interrupted system call should be restarted 2024-08-25T21:08:01.072 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-08-25T21:08:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-08-25T21:08:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:08:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local pool=mirror 2024-08-25T21:08:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local image=image-primary10 2024-08-25T21:08:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-08-25T21:08:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rbd --cluster cluster1 ls mirror 2024-08-25T21:08:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ grep -wq image-primary10 2024-08-25T21:08:01.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 0 2024-08-25T21:08:01.130 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -rf test-primary10 2024-08-25T21:08:01.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:OK 2024-08-25T21:08:01.132 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo OK 2024-08-25T21:08:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ cleanup 0 2024-08-25T21:08:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local error_code=0 2024-08-25T21:08:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ set +e 2024-08-25T21:08:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' 0 -ne 0 ']' 2024-08-25T21:08:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -z '' ']' 2024-08-25T21:08:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster instance 2024-08-25T21:08:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T21:08:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ceph --cluster cluster1 osd pool rm mirror mirror --yes-i-really-really-mean-it 2024-08-25T21:08:01.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi121.stderr:2024-08-25T21:08:01.950+0000 7f8bc2200640 -1 rbd::mirror::image_deleter::SnapshotPurgeRequest: 0x5593f53ee7e0 handle_snap_remove: failed to remove snapshot: (2) No such file or directory 2024-08-25T21:08:01.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi121.stderr:2024-08-25T21:08:01.952+0000 7f8ba0600640 -1 rbd::mirror::image_deleter::TrashRemoveRequest: 0x5593f63c0000 handle_purge_snapshots: failed to purge image snapshots: (2) No such file or directory 2024-08-25T21:08:02.052 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:pool 'mirror' does not exist 2024-08-25T21:08:02.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T21:08:02.067 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ceph --cluster cluster2 osd pool rm mirror mirror --yes-i-really-really-mean-it 2024-08-25T21:08:02.960 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:pool 'mirror' does not exist 2024-08-25T21:08:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T21:08:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ceph --cluster cluster1 osd pool rm mirror_parent mirror_parent --yes-i-really-really-mean-it 2024-08-25T21:08:04.136 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:pool 'mirror_parent' does not exist 2024-08-25T21:08:04.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ CEPH_ARGS= 2024-08-25T21:08:04.152 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ ceph --cluster cluster2 osd pool rm mirror_parent mirror_parent --yes-i-really-really-mean-it 2024-08-25T21:08:04.256 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi121.stderr:2024-08-25T21:08:04.255+0000 7f8847e00640 -1 rbd::mirror::LeaderWatcher: 0x555adefdaf00 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2024-08-25T21:08:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi121.stderr:2024-08-25T21:08:04.265+0000 7f8bc9a00640 -1 rbd::mirror::LeaderWatcher: 0x5593f5157500 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2024-08-25T21:08:04.976 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:pool 'mirror_parent' does not exist 2024-08-25T21:08:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for cluster in "${CLUSTER1}" "${CLUSTER2}" 2024-08-25T21:08:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ stop_mirrors cluster1 2024-08-25T21:08:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1 2024-08-25T21:08:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local sig= 2024-08-25T21:08:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ seq 0 0 2024-08-25T21:08:05.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-08-25T21:08:05.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ stop_mirror cluster1:0 '' 2024-08-25T21:08:05.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster1:0 2024-08-25T21:08:05.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local sig= 2024-08-25T21:08:05.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test -n 1 2024-08-25T21:08:05.016 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 2024-08-25T21:08:05.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for cluster in "${CLUSTER1}" "${CLUSTER2}" 2024-08-25T21:08:05.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ stop_mirrors cluster2 2024-08-25T21:08:05.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2 2024-08-25T21:08:05.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local sig= 2024-08-25T21:08:05.017 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:++ seq 0 0 2024-08-25T21:08:05.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-08-25T21:08:05.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ stop_mirror cluster2:0 '' 2024-08-25T21:08:05.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local cluster=cluster2:0 2024-08-25T21:08:05.019 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ local sig= 2024-08-25T21:08:05.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test -n 1 2024-08-25T21:08:05.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ return 2024-08-25T21:08:05.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' -z 1 ']' 2024-08-25T21:08:05.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ test '' = /tmp/tmp.vVBlul05Au 2024-08-25T21:08:05.020 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ rm -Rf /tmp/tmp.vVBlul05Au 2024-08-25T21:08:05.031 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stdout:OK 2024-08-25T21:08:05.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ '[' 0 -eq 0 ']' 2024-08-25T21:08:05.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ echo OK 2024-08-25T21:08:05.032 INFO:tasks.workunit.cluster1.client.mirror.smithi121.stderr:+ exit 0 2024-08-25T21:08:05.033 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-25T21:08:05.033 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp 2024-08-25T21:08:05.312 INFO:tasks.workunit:Stopping ['rbd/compare_mirror_images.sh'] on cluster1.client.mirror... 2024-08-25T21:08:05.312 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.cluster1.client.mirror /home/ubuntu/cephtest/clone.cluster1.client.mirror 2024-08-25T21:08:05.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi121.stderr:2024-08-25T21:08:05.498+0000 7f8bca400640 -1 rbd::mirror::LeaderWatcher: 0x5593f4eacf00 handle_notify_heartbeat: error notifying heartbeat: (2) No such file or directory, releasing leader 2024-08-25T21:08:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi121.stderr:2024-08-25T21:08:05.498+0000 7f8bc2200640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x5593f5158900 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-08-25T21:08:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi121.stderr:2024-08-25T21:08:05.498+0000 7f8bc2200640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x5593f5158240 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-08-25T21:08:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi121.stderr:2024-08-25T21:08:05.498+0000 7f8bc2200640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x5593f5458240 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-08-25T21:08:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi121.stderr:2024-08-25T21:08:05.499+0000 7f8bc1800640 -1 rbd::mirror::PoolWatcher: 0x5593f3f21680 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-08-25T21:08:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi121.stderr:2024-08-25T21:08:05.499+0000 7f8bc2200640 -1 rbd::mirror::PoolWatcher: 0x5593f52d23c0 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-08-25T21:08:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi121.stderr:2024-08-25T21:08:05.499+0000 7f8bc2200640 -1 rbd::mirror::PoolWatcher: 0x5593f52d2000 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-08-25T21:08:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi121.stderr:2024-08-25T21:08:05.500+0000 7f8bc9a00640 -1 rbd::mirror::LeaderWatcher: 0x5593f4eacf00 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2024-08-25T21:08:05.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi121.stderr:2024-08-25T21:08:05.500+0000 7f8bc9a00640 -1 rbd::mirror::LeaderWatcher: 0x5593f4eacf00 handle_release_leader_lock: error releasing lock: (2) No such file or directory 2024-08-25T21:08:06.419 DEBUG:teuthology.parallel:result is None 2024-08-25T21:08:06.419 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror 2024-08-25T21:08:06.590 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror 2024-08-25T21:08:06.590 DEBUG:teuthology.orchestra.run.smithi121:> rmdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2024-08-25T21:08:06.696 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror 2024-08-25T21:08:06.697 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2024-08-25T21:08:06.707 DEBUG:tasks.rbd_mirror.cluster2.client.mirror.0:waiting for process to exit 2024-08-25T21:08:06.707 INFO:teuthology.orchestra.run:waiting for 300 2024-08-25T21:08:06.731 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Stopped 2024-08-25T21:08:06.731 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2024-08-25T21:08:06.825 DEBUG:tasks.rbd_mirror.cluster1.client.mirror.0:waiting for process to exit 2024-08-25T21:08:06.826 INFO:teuthology.orchestra.run:waiting for 300 2024-08-25T21:08:06.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Stopped 2024-08-25T21:08:06.850 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-08-25T21:08:06.858 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2024-08-25T21:08:06.859 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-08-25T21:08:07.261 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T21:08:07.261 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-08-25T21:08:07.278 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":4377,"stamp":"2024-08-25T21:08:06.848154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2414448,"kb_used_data":16308,"kb_used_omap":4,"kb_used_meta":2398075,"kb_avail":280701072,"statfs":{"total":289910292480,"available":287437897728,"internally_reserved":0,"allocated":16699392,"data_stored":9716706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":2455629136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-299025112,"num_objects":-115,"num_object_clones":0,"num_object_copies":-230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-115,"num_whiteouts":0,"num_read":-516525,"num_read_kb":-36091553,"num_write":-132467,"num_write_kb":-29366846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-12,"num_bytes_recovered":-12632080,"num_keys_recovered":-2,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.041213"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3872,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.859157+0000","last_change":"2024-08-25T19:46:23.325343+0000","last_active":"2024-08-25T21:08:02.859157+0000","last_peered":"2024-08-25T21:08:02.859157+0000","last_clean":"2024-08-25T21:08:02.859157+0000","last_became_active":"2024-08-25T19:46:21.322148+0000","last_became_peered":"2024-08-25T19:46:21.322148+0000","last_unstale":"2024-08-25T21:08:02.859157+0000","last_undegraded":"2024-08-25T21:08:02.859157+0000","last_fullsized":"2024-08-25T21:08:02.859157+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:02:27.808835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010803760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3872,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.974365+0000","last_change":"2024-08-25T19:46:23.325326+0000","last_active":"2024-08-25T21:08:02.974365+0000","last_peered":"2024-08-25T21:08:02.974365+0000","last_clean":"2024-08-25T21:08:02.974365+0000","last_became_active":"2024-08-25T19:46:21.322277+0000","last_became_peered":"2024-08-25T19:46:21.322277+0000","last_unstale":"2024-08-25T21:08:02.974365+0000","last_undegraded":"2024-08-25T21:08:02.974365+0000","last_fullsized":"2024-08-25T21:08:02.974365+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:33:07.769968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098805500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3874,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.975236+0000","last_change":"2024-08-25T19:46:23.325616+0000","last_active":"2024-08-25T21:08:02.975236+0000","last_peered":"2024-08-25T21:08:02.975236+0000","last_clean":"2024-08-25T21:08:02.975236+0000","last_became_active":"2024-08-25T19:46:21.322099+0000","last_became_peered":"2024-08-25T19:46:21.322099+0000","last_unstale":"2024-08-25T21:08:02.975236+0000","last_undegraded":"2024-08-25T21:08:02.975236+0000","last_fullsized":"2024-08-25T21:08:02.975236+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T03:29:14.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248755,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3874,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.859012+0000","last_change":"2024-08-25T19:46:23.324883+0000","last_active":"2024-08-25T21:08:02.859012+0000","last_peered":"2024-08-25T21:08:02.859012+0000","last_clean":"2024-08-25T21:08:02.859012+0000","last_became_active":"2024-08-25T19:46:21.322641+0000","last_became_peered":"2024-08-25T19:46:21.322641+0000","last_unstale":"2024-08-25T21:08:02.859012+0000","last_undegraded":"2024-08-25T21:08:02.859012+0000","last_fullsized":"2024-08-25T21:08:02.859012+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T07:29:32.662242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062717600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":3876,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:03.330032+0000","last_change":"2024-08-25T19:46:23.319590+0000","last_active":"2024-08-25T21:08:03.330032+0000","last_peered":"2024-08-25T21:08:03.330032+0000","last_clean":"2024-08-25T21:08:03.330032+0000","last_became_active":"2024-08-25T19:46:21.321161+0000","last_became_peered":"2024-08-25T19:46:21.321161+0000","last_unstale":"2024-08-25T21:08:03.330032+0000","last_undegraded":"2024-08-25T21:08:03.330032+0000","last_fullsized":"2024-08-25T21:08:03.330032+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T20:13:55.940616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056347000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3877,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.870911+0000","last_change":"2024-08-25T19:46:23.351559+0000","last_active":"2024-08-25T21:08:04.870911+0000","last_peered":"2024-08-25T21:08:04.870911+0000","last_clean":"2024-08-25T21:08:04.870911+0000","last_became_active":"2024-08-25T19:46:21.320531+0000","last_became_peered":"2024-08-25T19:46:21.320531+0000","last_unstale":"2024-08-25T21:08:04.870911+0000","last_undegraded":"2024-08-25T21:08:04.870911+0000","last_fullsized":"2024-08-25T21:08:04.870911+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T20:13:55.940616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046501600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3878,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.866536+0000","last_change":"2024-08-25T19:46:23.351749+0000","last_active":"2024-08-25T21:08:04.866536+0000","last_peered":"2024-08-25T21:08:04.866536+0000","last_clean":"2024-08-25T21:08:04.866536+0000","last_became_active":"2024-08-25T19:46:21.319377+0000","last_became_peered":"2024-08-25T19:46:21.319377+0000","last_unstale":"2024-08-25T21:08:04.866536+0000","last_undegraded":"2024-08-25T21:08:04.866536+0000","last_fullsized":"2024-08-25T21:08:04.866536+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:06:47.731173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064495999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":3875,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.975337+0000","last_change":"2024-08-25T19:46:23.324873+0000","last_active":"2024-08-25T21:08:02.975337+0000","last_peered":"2024-08-25T21:08:02.975337+0000","last_clean":"2024-08-25T21:08:02.975337+0000","last_became_active":"2024-08-25T19:46:21.318722+0000","last_became_peered":"2024-08-25T19:46:21.318722+0000","last_unstale":"2024-08-25T21:08:02.975337+0000","last_undegraded":"2024-08-25T21:08:02.975337+0000","last_fullsized":"2024-08-25T21:08:02.975337+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:29:54.480849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099496299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":3943,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.976424+0000","last_change":"2024-08-25T19:46:17.304647+0000","last_active":"2024-08-25T21:08:02.976424+0000","last_peered":"2024-08-25T21:08:02.976424+0000","last_clean":"2024-08-25T21:08:02.976424+0000","last_became_active":"2024-08-25T19:46:17.304344+0000","last_became_peered":"2024-08-25T19:46:17.304344+0000","last_unstale":"2024-08-25T21:08:02.976424+0000","last_undegraded":"2024-08-25T21:08:02.976424+0000","last_fullsized":"2024-08-25T21:08:02.976424+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:16.293123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:16.293123+0000","last_clean_scrub_stamp":"2024-08-25T19:46:16.293123+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T04:35:50.579834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739353,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708172,"kb_used_data":5048,"kb_used_omap":1,"kb_used_meta":703102,"kb_avail":93663668,"statfs":{"total":96636764160,"available":95911596032,"internally_reserved":0,"allocated":5169152,"data_stored":2845317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":719976901},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739354,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798996,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":793342,"kb_avail":93572844,"statfs":{"total":96636764160,"available":95818592256,"internally_reserved":0,"allocated":5767168,"data_stored":3435704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":812382661},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739353,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907280,"kb_used_data":5628,"kb_used_omap":1,"kb_used_meta":901630,"kb_avail":93464560,"statfs":{"total":96636764160,"available":95707709440,"internally_reserved":0,"allocated":5763072,"data_stored":3435685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":923269574},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T21:08:07.279 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-08-25T21:08:07.625 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T21:08:07.625 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-08-25T21:08:07.642 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":4377,"stamp":"2024-08-25T21:08:06.848154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2414448,"kb_used_data":16308,"kb_used_omap":4,"kb_used_meta":2398075,"kb_avail":280701072,"statfs":{"total":289910292480,"available":287437897728,"internally_reserved":0,"allocated":16699392,"data_stored":9716706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":2455629136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-299025112,"num_objects":-115,"num_object_clones":0,"num_object_copies":-230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-115,"num_whiteouts":0,"num_read":-516525,"num_read_kb":-36091553,"num_write":-132467,"num_write_kb":-29366846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-12,"num_bytes_recovered":-12632080,"num_keys_recovered":-2,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.041213"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3872,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.859157+0000","last_change":"2024-08-25T19:46:23.325343+0000","last_active":"2024-08-25T21:08:02.859157+0000","last_peered":"2024-08-25T21:08:02.859157+0000","last_clean":"2024-08-25T21:08:02.859157+0000","last_became_active":"2024-08-25T19:46:21.322148+0000","last_became_peered":"2024-08-25T19:46:21.322148+0000","last_unstale":"2024-08-25T21:08:02.859157+0000","last_undegraded":"2024-08-25T21:08:02.859157+0000","last_fullsized":"2024-08-25T21:08:02.859157+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:02:27.808835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010803760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3872,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.974365+0000","last_change":"2024-08-25T19:46:23.325326+0000","last_active":"2024-08-25T21:08:02.974365+0000","last_peered":"2024-08-25T21:08:02.974365+0000","last_clean":"2024-08-25T21:08:02.974365+0000","last_became_active":"2024-08-25T19:46:21.322277+0000","last_became_peered":"2024-08-25T19:46:21.322277+0000","last_unstale":"2024-08-25T21:08:02.974365+0000","last_undegraded":"2024-08-25T21:08:02.974365+0000","last_fullsized":"2024-08-25T21:08:02.974365+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:33:07.769968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098805500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3874,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.975236+0000","last_change":"2024-08-25T19:46:23.325616+0000","last_active":"2024-08-25T21:08:02.975236+0000","last_peered":"2024-08-25T21:08:02.975236+0000","last_clean":"2024-08-25T21:08:02.975236+0000","last_became_active":"2024-08-25T19:46:21.322099+0000","last_became_peered":"2024-08-25T19:46:21.322099+0000","last_unstale":"2024-08-25T21:08:02.975236+0000","last_undegraded":"2024-08-25T21:08:02.975236+0000","last_fullsized":"2024-08-25T21:08:02.975236+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T03:29:14.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248755,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3874,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.859012+0000","last_change":"2024-08-25T19:46:23.324883+0000","last_active":"2024-08-25T21:08:02.859012+0000","last_peered":"2024-08-25T21:08:02.859012+0000","last_clean":"2024-08-25T21:08:02.859012+0000","last_became_active":"2024-08-25T19:46:21.322641+0000","last_became_peered":"2024-08-25T19:46:21.322641+0000","last_unstale":"2024-08-25T21:08:02.859012+0000","last_undegraded":"2024-08-25T21:08:02.859012+0000","last_fullsized":"2024-08-25T21:08:02.859012+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T07:29:32.662242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062717600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":3876,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:03.330032+0000","last_change":"2024-08-25T19:46:23.319590+0000","last_active":"2024-08-25T21:08:03.330032+0000","last_peered":"2024-08-25T21:08:03.330032+0000","last_clean":"2024-08-25T21:08:03.330032+0000","last_became_active":"2024-08-25T19:46:21.321161+0000","last_became_peered":"2024-08-25T19:46:21.321161+0000","last_unstale":"2024-08-25T21:08:03.330032+0000","last_undegraded":"2024-08-25T21:08:03.330032+0000","last_fullsized":"2024-08-25T21:08:03.330032+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T20:13:55.940616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056347000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3877,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.870911+0000","last_change":"2024-08-25T19:46:23.351559+0000","last_active":"2024-08-25T21:08:04.870911+0000","last_peered":"2024-08-25T21:08:04.870911+0000","last_clean":"2024-08-25T21:08:04.870911+0000","last_became_active":"2024-08-25T19:46:21.320531+0000","last_became_peered":"2024-08-25T19:46:21.320531+0000","last_unstale":"2024-08-25T21:08:04.870911+0000","last_undegraded":"2024-08-25T21:08:04.870911+0000","last_fullsized":"2024-08-25T21:08:04.870911+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T20:13:55.940616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046501600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3878,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.866536+0000","last_change":"2024-08-25T19:46:23.351749+0000","last_active":"2024-08-25T21:08:04.866536+0000","last_peered":"2024-08-25T21:08:04.866536+0000","last_clean":"2024-08-25T21:08:04.866536+0000","last_became_active":"2024-08-25T19:46:21.319377+0000","last_became_peered":"2024-08-25T19:46:21.319377+0000","last_unstale":"2024-08-25T21:08:04.866536+0000","last_undegraded":"2024-08-25T21:08:04.866536+0000","last_fullsized":"2024-08-25T21:08:04.866536+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:06:47.731173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064495999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":3875,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.975337+0000","last_change":"2024-08-25T19:46:23.324873+0000","last_active":"2024-08-25T21:08:02.975337+0000","last_peered":"2024-08-25T21:08:02.975337+0000","last_clean":"2024-08-25T21:08:02.975337+0000","last_became_active":"2024-08-25T19:46:21.318722+0000","last_became_peered":"2024-08-25T19:46:21.318722+0000","last_unstale":"2024-08-25T21:08:02.975337+0000","last_undegraded":"2024-08-25T21:08:02.975337+0000","last_fullsized":"2024-08-25T21:08:02.975337+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:29:54.480849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099496299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":3943,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.976424+0000","last_change":"2024-08-25T19:46:17.304647+0000","last_active":"2024-08-25T21:08:02.976424+0000","last_peered":"2024-08-25T21:08:02.976424+0000","last_clean":"2024-08-25T21:08:02.976424+0000","last_became_active":"2024-08-25T19:46:17.304344+0000","last_became_peered":"2024-08-25T19:46:17.304344+0000","last_unstale":"2024-08-25T21:08:02.976424+0000","last_undegraded":"2024-08-25T21:08:02.976424+0000","last_fullsized":"2024-08-25T21:08:02.976424+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:16.293123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:16.293123+0000","last_clean_scrub_stamp":"2024-08-25T19:46:16.293123+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T04:35:50.579834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739353,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708172,"kb_used_data":5048,"kb_used_omap":1,"kb_used_meta":703102,"kb_avail":93663668,"statfs":{"total":96636764160,"available":95911596032,"internally_reserved":0,"allocated":5169152,"data_stored":2845317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":719976901},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739354,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798996,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":793342,"kb_avail":93572844,"statfs":{"total":96636764160,"available":95818592256,"internally_reserved":0,"allocated":5767168,"data_stored":3435704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":812382661},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739353,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907280,"kb_used_data":5628,"kb_used_omap":1,"kb_used_meta":901630,"kb_avail":93464560,"statfs":{"total":96636764160,"available":95707709440,"internally_reserved":0,"allocated":5763072,"data_stored":3435685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":923269574},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T21:08:07.643 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2024-08-25T21:08:07.643 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-08-25T21:08:07.996 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T21:08:07.997 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-08-25T21:08:08.014 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":4377,"stamp":"2024-08-25T21:08:06.848154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2414448,"kb_used_data":16308,"kb_used_omap":4,"kb_used_meta":2398075,"kb_avail":280701072,"statfs":{"total":289910292480,"available":287437897728,"internally_reserved":0,"allocated":16699392,"data_stored":9716706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":2455629136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-299025112,"num_objects":-115,"num_object_clones":0,"num_object_copies":-230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-115,"num_whiteouts":0,"num_read":-516525,"num_read_kb":-36091553,"num_write":-132467,"num_write_kb":-29366846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-12,"num_bytes_recovered":-12632080,"num_keys_recovered":-2,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.041213"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3872,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.859157+0000","last_change":"2024-08-25T19:46:23.325343+0000","last_active":"2024-08-25T21:08:02.859157+0000","last_peered":"2024-08-25T21:08:02.859157+0000","last_clean":"2024-08-25T21:08:02.859157+0000","last_became_active":"2024-08-25T19:46:21.322148+0000","last_became_peered":"2024-08-25T19:46:21.322148+0000","last_unstale":"2024-08-25T21:08:02.859157+0000","last_undegraded":"2024-08-25T21:08:02.859157+0000","last_fullsized":"2024-08-25T21:08:02.859157+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:02:27.808835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010803760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3872,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.974365+0000","last_change":"2024-08-25T19:46:23.325326+0000","last_active":"2024-08-25T21:08:02.974365+0000","last_peered":"2024-08-25T21:08:02.974365+0000","last_clean":"2024-08-25T21:08:02.974365+0000","last_became_active":"2024-08-25T19:46:21.322277+0000","last_became_peered":"2024-08-25T19:46:21.322277+0000","last_unstale":"2024-08-25T21:08:02.974365+0000","last_undegraded":"2024-08-25T21:08:02.974365+0000","last_fullsized":"2024-08-25T21:08:02.974365+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:33:07.769968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098805500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3874,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.975236+0000","last_change":"2024-08-25T19:46:23.325616+0000","last_active":"2024-08-25T21:08:02.975236+0000","last_peered":"2024-08-25T21:08:02.975236+0000","last_clean":"2024-08-25T21:08:02.975236+0000","last_became_active":"2024-08-25T19:46:21.322099+0000","last_became_peered":"2024-08-25T19:46:21.322099+0000","last_unstale":"2024-08-25T21:08:02.975236+0000","last_undegraded":"2024-08-25T21:08:02.975236+0000","last_fullsized":"2024-08-25T21:08:02.975236+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T03:29:14.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248755,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3874,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.859012+0000","last_change":"2024-08-25T19:46:23.324883+0000","last_active":"2024-08-25T21:08:02.859012+0000","last_peered":"2024-08-25T21:08:02.859012+0000","last_clean":"2024-08-25T21:08:02.859012+0000","last_became_active":"2024-08-25T19:46:21.322641+0000","last_became_peered":"2024-08-25T19:46:21.322641+0000","last_unstale":"2024-08-25T21:08:02.859012+0000","last_undegraded":"2024-08-25T21:08:02.859012+0000","last_fullsized":"2024-08-25T21:08:02.859012+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T07:29:32.662242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062717600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":3876,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:03.330032+0000","last_change":"2024-08-25T19:46:23.319590+0000","last_active":"2024-08-25T21:08:03.330032+0000","last_peered":"2024-08-25T21:08:03.330032+0000","last_clean":"2024-08-25T21:08:03.330032+0000","last_became_active":"2024-08-25T19:46:21.321161+0000","last_became_peered":"2024-08-25T19:46:21.321161+0000","last_unstale":"2024-08-25T21:08:03.330032+0000","last_undegraded":"2024-08-25T21:08:03.330032+0000","last_fullsized":"2024-08-25T21:08:03.330032+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T20:13:55.940616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056347000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3877,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.870911+0000","last_change":"2024-08-25T19:46:23.351559+0000","last_active":"2024-08-25T21:08:04.870911+0000","last_peered":"2024-08-25T21:08:04.870911+0000","last_clean":"2024-08-25T21:08:04.870911+0000","last_became_active":"2024-08-25T19:46:21.320531+0000","last_became_peered":"2024-08-25T19:46:21.320531+0000","last_unstale":"2024-08-25T21:08:04.870911+0000","last_undegraded":"2024-08-25T21:08:04.870911+0000","last_fullsized":"2024-08-25T21:08:04.870911+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T20:13:55.940616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046501600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3878,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.866536+0000","last_change":"2024-08-25T19:46:23.351749+0000","last_active":"2024-08-25T21:08:04.866536+0000","last_peered":"2024-08-25T21:08:04.866536+0000","last_clean":"2024-08-25T21:08:04.866536+0000","last_became_active":"2024-08-25T19:46:21.319377+0000","last_became_peered":"2024-08-25T19:46:21.319377+0000","last_unstale":"2024-08-25T21:08:04.866536+0000","last_undegraded":"2024-08-25T21:08:04.866536+0000","last_fullsized":"2024-08-25T21:08:04.866536+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:06:47.731173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064495999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":3875,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.975337+0000","last_change":"2024-08-25T19:46:23.324873+0000","last_active":"2024-08-25T21:08:02.975337+0000","last_peered":"2024-08-25T21:08:02.975337+0000","last_clean":"2024-08-25T21:08:02.975337+0000","last_became_active":"2024-08-25T19:46:21.318722+0000","last_became_peered":"2024-08-25T19:46:21.318722+0000","last_unstale":"2024-08-25T21:08:02.975337+0000","last_undegraded":"2024-08-25T21:08:02.975337+0000","last_fullsized":"2024-08-25T21:08:02.975337+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:29:54.480849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099496299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":3943,"reported_epoch":1923,"state":"active+clean","last_fresh":"2024-08-25T21:08:02.976424+0000","last_change":"2024-08-25T19:46:17.304647+0000","last_active":"2024-08-25T21:08:02.976424+0000","last_peered":"2024-08-25T21:08:02.976424+0000","last_clean":"2024-08-25T21:08:02.976424+0000","last_became_active":"2024-08-25T19:46:17.304344+0000","last_became_peered":"2024-08-25T19:46:17.304344+0000","last_unstale":"2024-08-25T21:08:02.976424+0000","last_undegraded":"2024-08-25T21:08:02.976424+0000","last_fullsized":"2024-08-25T21:08:02.976424+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:16.293123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:16.293123+0000","last_clean_scrub_stamp":"2024-08-25T19:46:16.293123+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T04:35:50.579834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739353,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708172,"kb_used_data":5048,"kb_used_omap":1,"kb_used_meta":703102,"kb_avail":93663668,"statfs":{"total":96636764160,"available":95911596032,"internally_reserved":0,"allocated":5169152,"data_stored":2845317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":719976901},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739354,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798996,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":793342,"kb_avail":93572844,"statfs":{"total":96636764160,"available":95818592256,"internally_reserved":0,"allocated":5767168,"data_stored":3435704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":812382661},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739353,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907280,"kb_used_data":5628,"kb_used_omap":1,"kb_used_meta":901630,"kb_avail":93464560,"statfs":{"total":96636764160,"available":95707709440,"internally_reserved":0,"allocated":5763072,"data_stored":3435685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":923269574},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T21:08:08.015 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2024-08-25T21:08:08.367 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T21:08:08.368 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1924,"fsid":"2c473bc9-1901-47c2-9407-3117e308ba4c","created":"2024-08-25T19:46:04.378052+0000","modified":"2024-08-25T21:08:04.855107+0000","last_up_change":"2024-08-25T19:46:15.292315+0000","last_in_change":"2024-08-25T19:46:05.510002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-25T19:46:15.482233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-25T19:46:20.072680+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b6d14b3-fc24-424d-978d-f031d2571911","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6809","nonce":1885352393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6811","nonce":1885352393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6815","nonce":1885352393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1885352393},{"type":"v1","addr":"172.21.15.121:6813","nonce":1885352393}]},"public_addr":"172.21.15.121:6809/1885352393","cluster_addr":"172.21.15.121:6811/1885352393","heartbeat_back_addr":"172.21.15.121:6815/1885352393","heartbeat_front_addr":"172.21.15.121:6813/1885352393","state":["exists","up"]},{"osd":1,"uuid":"8139be87-fa16-4528-be86-d917a6662414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6801","nonce":2034030038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6803","nonce":2034030038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6807","nonce":2034030038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2034030038},{"type":"v1","addr":"172.21.15.121:6805","nonce":2034030038}]},"public_addr":"172.21.15.121:6801/2034030038","cluster_addr":"172.21.15.121:6803/2034030038","heartbeat_back_addr":"172.21.15.121:6807/2034030038","heartbeat_front_addr":"172.21.15.121:6805/2034030038","state":["exists","up"]},{"osd":2,"uuid":"3275c148-71c8-4157-8a72-325fa4bcff38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6817","nonce":4005813664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6819","nonce":4005813664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6823","nonce":4005813664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4005813664},{"type":"v1","addr":"172.21.15.121:6821","nonce":4005813664}]},"public_addr":"172.21.15.121:6817/4005813664","cluster_addr":"172.21.15.121:6819/4005813664","heartbeat_back_addr":"172.21.15.121:6823/4005813664","heartbeat_front_addr":"172.21.15.121:6821/4005813664","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:46:12.024441+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:46:11.836435+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:46:12.306093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":3246,"length":1},{"begin":3250,"length":1},{"begin":3257,"length":1},{"begin":3265,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T21:08:09.386 INFO:tasks.ceph:Scrubbing cluster2.osd.0 2024-08-25T21:08:09.386 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-08-25T21:08:09.597 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-08-25T21:08:09.597 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-25T21:08:09.597 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-08-25T21:08:09.612 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd deep-scrub 0 2024-08-25T21:08:09.961 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 0 to deep-scrub 2024-08-25T21:08:09.979 INFO:tasks.ceph:Scrubbing cluster2.osd.1 2024-08-25T21:08:09.979 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-08-25T21:08:10.158 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-08-25T21:08:10.158 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-25T21:08:10.158 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-08-25T21:08:10.173 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd deep-scrub 1 2024-08-25T21:08:10.520 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 1 to deep-scrub 2024-08-25T21:08:10.538 INFO:tasks.ceph:Scrubbing cluster2.osd.2 2024-08-25T21:08:10.539 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-08-25T21:08:10.717 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-08-25T21:08:10.717 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-25T21:08:10.717 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-08-25T21:08:10.732 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd deep-scrub 2 2024-08-25T21:08:11.084 INFO:teuthology.orchestra.run.smithi121.stderr:instructed osd(s) 2 to deep-scrub 2024-08-25T21:08:11.103 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-08-25T21:08:11.454 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T21:08:11.455 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-08-25T21:08:11.472 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":4379,"stamp":"2024-08-25T21:08:10.848872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56,"ondisk_log_size":56,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2418636,"kb_used_data":16336,"kb_used_omap":4,"kb_used_meta":2402235,"kb_avail":280696884,"statfs":{"total":289910292480,"available":287433609216,"internally_reserved":0,"allocated":16728064,"data_stored":9727786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":2459888976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-299025112,"num_objects":-115,"num_object_clones":0,"num_object_copies":-230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-115,"num_whiteouts":0,"num_read":-516525,"num_read_kb":-36091553,"num_write":-132467,"num_write_kb":-29366846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-12,"num_bytes_recovered":-12632080,"num_keys_recovered":-2,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001355"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3875,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:06.466386+0000","last_change":"2024-08-25T19:46:23.325343+0000","last_active":"2024-08-25T21:08:06.466386+0000","last_peered":"2024-08-25T21:08:06.466386+0000","last_clean":"2024-08-25T21:08:06.466386+0000","last_became_active":"2024-08-25T19:46:21.322148+0000","last_became_peered":"2024-08-25T19:46:21.322148+0000","last_unstale":"2024-08-25T21:08:06.466386+0000","last_undegraded":"2024-08-25T21:08:06.466386+0000","last_fullsized":"2024-08-25T21:08:06.466386+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010803760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3875,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:06.467625+0000","last_change":"2024-08-25T19:46:23.325326+0000","last_active":"2024-08-25T21:08:06.467625+0000","last_peered":"2024-08-25T21:08:06.467625+0000","last_clean":"2024-08-25T21:08:06.467625+0000","last_became_active":"2024-08-25T19:46:21.322277+0000","last_became_peered":"2024-08-25T19:46:21.322277+0000","last_unstale":"2024-08-25T21:08:06.467625+0000","last_undegraded":"2024-08-25T21:08:06.467625+0000","last_fullsized":"2024-08-25T21:08:06.467625+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098805500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3877,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:06.467067+0000","last_change":"2024-08-25T19:46:23.325616+0000","last_active":"2024-08-25T21:08:06.467067+0000","last_peered":"2024-08-25T21:08:06.467067+0000","last_clean":"2024-08-25T21:08:06.467067+0000","last_became_active":"2024-08-25T19:46:21.322099+0000","last_became_peered":"2024-08-25T19:46:21.322099+0000","last_unstale":"2024-08-25T21:08:06.467067+0000","last_undegraded":"2024-08-25T21:08:06.467067+0000","last_fullsized":"2024-08-25T21:08:06.467067+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248755,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3877,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:06.467069+0000","last_change":"2024-08-25T19:46:23.324883+0000","last_active":"2024-08-25T21:08:06.467069+0000","last_peered":"2024-08-25T21:08:06.467069+0000","last_clean":"2024-08-25T21:08:06.467069+0000","last_became_active":"2024-08-25T19:46:21.322641+0000","last_became_peered":"2024-08-25T19:46:21.322641+0000","last_unstale":"2024-08-25T21:08:06.467069+0000","last_undegraded":"2024-08-25T21:08:06.467069+0000","last_fullsized":"2024-08-25T21:08:06.467069+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062717600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"1924'3","reported_seq":3888,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:10.055002+0000","last_change":"2024-08-25T21:08:10.055028+0000","last_active":"2024-08-25T21:08:10.055002+0000","last_peered":"2024-08-25T21:08:10.055002+0000","last_clean":"2024-08-25T21:08:10.055002+0000","last_became_active":"2024-08-25T19:46:21.321161+0000","last_became_peered":"2024-08-25T19:46:21.321161+0000","last_unstale":"2024-08-25T21:08:10.055002+0000","last_undegraded":"2024-08-25T21:08:10.055002+0000","last_fullsized":"2024-08-25T21:08:10.055002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"1924'3","last_scrub_stamp":"2024-08-25T21:08:10.054918+0000","last_deep_scrub":"1924'3","last_deep_scrub_stamp":"2024-08-25T21:08:10.054918+0000","last_clean_scrub_stamp":"2024-08-25T21:08:10.054918+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:33:58.773660+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00056347000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3877,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.870911+0000","last_change":"2024-08-25T19:46:23.351559+0000","last_active":"2024-08-25T21:08:04.870911+0000","last_peered":"2024-08-25T21:08:04.870911+0000","last_clean":"2024-08-25T21:08:04.870911+0000","last_became_active":"2024-08-25T19:46:21.320531+0000","last_became_peered":"2024-08-25T19:46:21.320531+0000","last_unstale":"2024-08-25T21:08:04.870911+0000","last_undegraded":"2024-08-25T21:08:04.870911+0000","last_fullsized":"2024-08-25T21:08:04.870911+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T20:13:55.940616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046501600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3878,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.866536+0000","last_change":"2024-08-25T19:46:23.351749+0000","last_active":"2024-08-25T21:08:04.866536+0000","last_peered":"2024-08-25T21:08:04.866536+0000","last_clean":"2024-08-25T21:08:04.866536+0000","last_became_active":"2024-08-25T19:46:21.319377+0000","last_became_peered":"2024-08-25T19:46:21.319377+0000","last_unstale":"2024-08-25T21:08:04.866536+0000","last_undegraded":"2024-08-25T21:08:04.866536+0000","last_fullsized":"2024-08-25T21:08:04.866536+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:06:47.731173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064495999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":3878,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:06.460956+0000","last_change":"2024-08-25T19:46:23.324873+0000","last_active":"2024-08-25T21:08:06.460956+0000","last_peered":"2024-08-25T21:08:06.460956+0000","last_clean":"2024-08-25T21:08:06.460956+0000","last_became_active":"2024-08-25T19:46:21.318722+0000","last_became_peered":"2024-08-25T19:46:21.318722+0000","last_unstale":"2024-08-25T21:08:06.460956+0000","last_undegraded":"2024-08-25T21:08:06.460956+0000","last_fullsized":"2024-08-25T21:08:06.460956+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:20.306525+0000","last_clean_scrub_stamp":"2024-08-25T19:46:20.306525+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099496299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":3946,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:06.463153+0000","last_change":"2024-08-25T19:46:17.304647+0000","last_active":"2024-08-25T21:08:06.463153+0000","last_peered":"2024-08-25T21:08:06.463153+0000","last_clean":"2024-08-25T21:08:06.463153+0000","last_became_active":"2024-08-25T19:46:17.304344+0000","last_became_peered":"2024-08-25T19:46:17.304344+0000","last_unstale":"2024-08-25T21:08:06.463153+0000","last_undegraded":"2024-08-25T21:08:06.463153+0000","last_fullsized":"2024-08-25T21:08:06.463153+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:46:16.293123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:46:16.293123+0000","last_clean_scrub_stamp":"2024-08-25T19:46:16.293123+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739354,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708184,"kb_used_data":5056,"kb_used_omap":1,"kb_used_meta":703102,"kb_avail":93663656,"statfs":{"total":96636764160,"available":95911583744,"internally_reserved":0,"allocated":5177344,"data_stored":2849004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":719976901},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739355,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803160,"kb_used_data":5640,"kb_used_omap":1,"kb_used_meta":797502,"kb_avail":93568680,"statfs":{"total":96636764160,"available":95814328320,"internally_reserved":0,"allocated":5775360,"data_stored":3439391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":816642501},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739354,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907292,"kb_used_data":5640,"kb_used_omap":1,"kb_used_meta":901630,"kb_avail":93464548,"statfs":{"total":96636764160,"available":95707697152,"internally_reserved":0,"allocated":5775360,"data_stored":3439391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":923269574},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T21:08:11.476 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-08-25T19:46:20.306525+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=46, tm_sec=20, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=8, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:11.476 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-08-25T19:46:20.306525+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=46, tm_sec=20, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=8, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:11.476 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-08-25T19:46:20.306525+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=46, tm_sec=20, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=8, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:11.476 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-08-25T19:46:20.306525+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=46, tm_sec=20, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=8, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:11.476 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-08-25T19:46:20.306525+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=46, tm_sec=20, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=8, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:11.477 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-08-25T19:46:20.306525+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=46, tm_sec=20, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=8, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:11.477 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-08-25T19:46:20.306525+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=46, tm_sec=20, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=8, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:11.477 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-08-25T19:46:16.293123+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=46, tm_sec=16, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=8, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:11.477 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-08-25T21:08:31.478 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-08-25T21:08:31.835 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-08-25T21:08:31.835 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-08-25T21:08:31.853 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":4389,"stamp":"2024-08-25T21:08:30.852992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59,"ondisk_log_size":59,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2418624,"kb_used_data":16336,"kb_used_omap":4,"kb_used_meta":2402235,"kb_avail":280696896,"statfs":{"total":289910292480,"available":287433621504,"internally_reserved":0,"allocated":16728064,"data_stored":9727786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":2459888976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002492"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3883,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:16.468708+0000","last_change":"2024-08-25T21:08:16.468754+0000","last_active":"2024-08-25T21:08:16.468708+0000","last_peered":"2024-08-25T21:08:16.468708+0000","last_clean":"2024-08-25T21:08:16.468708+0000","last_became_active":"2024-08-25T19:46:21.322148+0000","last_became_peered":"2024-08-25T19:46:21.322148+0000","last_unstale":"2024-08-25T21:08:16.468708+0000","last_undegraded":"2024-08-25T21:08:16.468708+0000","last_fullsized":"2024-08-25T21:08:16.468708+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T21:08:16.468603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T21:08:16.468603+0000","last_clean_scrub_stamp":"2024-08-25T21:08:16.468603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T01:46:53.760937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010803760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3883,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:13.463841+0000","last_change":"2024-08-25T21:08:13.463866+0000","last_active":"2024-08-25T21:08:13.463841+0000","last_peered":"2024-08-25T21:08:13.463841+0000","last_clean":"2024-08-25T21:08:13.463841+0000","last_became_active":"2024-08-25T19:46:21.322277+0000","last_became_peered":"2024-08-25T19:46:21.322277+0000","last_unstale":"2024-08-25T21:08:13.463841+0000","last_undegraded":"2024-08-25T21:08:13.463841+0000","last_fullsized":"2024-08-25T21:08:13.463841+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T21:08:13.463734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T21:08:13.463734+0000","last_clean_scrub_stamp":"2024-08-25T21:08:13.463734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T22:23:16.019982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098805500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3885,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:15.478871+0000","last_change":"2024-08-25T21:08:15.478935+0000","last_active":"2024-08-25T21:08:15.478871+0000","last_peered":"2024-08-25T21:08:15.478871+0000","last_clean":"2024-08-25T21:08:15.478871+0000","last_became_active":"2024-08-25T19:46:21.322099+0000","last_became_peered":"2024-08-25T19:46:21.322099+0000","last_unstale":"2024-08-25T21:08:15.478871+0000","last_undegraded":"2024-08-25T21:08:15.478871+0000","last_fullsized":"2024-08-25T21:08:15.478871+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T21:08:15.478727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T21:08:15.478727+0000","last_clean_scrub_stamp":"2024-08-25T21:08:15.478727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:56:54.171070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248755,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3885,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:14.501038+0000","last_change":"2024-08-25T21:08:14.501062+0000","last_active":"2024-08-25T21:08:14.501038+0000","last_peered":"2024-08-25T21:08:14.501038+0000","last_clean":"2024-08-25T21:08:14.501038+0000","last_became_active":"2024-08-25T19:46:21.322641+0000","last_became_peered":"2024-08-25T19:46:21.322641+0000","last_unstale":"2024-08-25T21:08:14.501038+0000","last_undegraded":"2024-08-25T21:08:14.501038+0000","last_fullsized":"2024-08-25T21:08:14.501038+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T21:08:14.500949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T21:08:14.500949+0000","last_clean_scrub_stamp":"2024-08-25T21:08:14.500949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:02:28.217620+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00062717600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"1924'3","reported_seq":3888,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:10.055002+0000","last_change":"2024-08-25T21:08:10.055028+0000","last_active":"2024-08-25T21:08:10.055002+0000","last_peered":"2024-08-25T21:08:10.055002+0000","last_clean":"2024-08-25T21:08:10.055002+0000","last_became_active":"2024-08-25T19:46:21.321161+0000","last_became_peered":"2024-08-25T19:46:21.321161+0000","last_unstale":"2024-08-25T21:08:10.055002+0000","last_undegraded":"2024-08-25T21:08:10.055002+0000","last_fullsized":"2024-08-25T21:08:10.055002+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"1924'3","last_scrub_stamp":"2024-08-25T21:08:10.054918+0000","last_deep_scrub":"1924'3","last_deep_scrub_stamp":"2024-08-25T21:08:10.054918+0000","last_clean_scrub_stamp":"2024-08-25T21:08:10.054918+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:33:58.773660+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00056347000000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3886,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:11.784731+0000","last_change":"2024-08-25T21:08:11.784751+0000","last_active":"2024-08-25T21:08:11.784731+0000","last_peered":"2024-08-25T21:08:11.784731+0000","last_clean":"2024-08-25T21:08:11.784731+0000","last_became_active":"2024-08-25T19:46:21.320531+0000","last_became_peered":"2024-08-25T19:46:21.320531+0000","last_unstale":"2024-08-25T21:08:11.784731+0000","last_undegraded":"2024-08-25T21:08:11.784731+0000","last_fullsized":"2024-08-25T21:08:11.784731+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T21:08:11.784627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T21:08:11.784627+0000","last_clean_scrub_stamp":"2024-08-25T21:08:11.784627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T02:57:10.229585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046501600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3887,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:12.744171+0000","last_change":"2024-08-25T21:08:12.744195+0000","last_active":"2024-08-25T21:08:12.744171+0000","last_peered":"2024-08-25T21:08:12.744171+0000","last_clean":"2024-08-25T21:08:12.744171+0000","last_became_active":"2024-08-25T19:46:21.319377+0000","last_became_peered":"2024-08-25T19:46:21.319377+0000","last_unstale":"2024-08-25T21:08:12.744171+0000","last_undegraded":"2024-08-25T21:08:12.744171+0000","last_fullsized":"2024-08-25T21:08:12.744171+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T21:08:12.744068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T21:08:12.744068+0000","last_clean_scrub_stamp":"2024-08-25T21:08:12.744068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:33:49.024697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064495999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"1924'2","reported_seq":3887,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:12.436984+0000","last_change":"2024-08-25T21:08:12.437015+0000","last_active":"2024-08-25T21:08:12.436984+0000","last_peered":"2024-08-25T21:08:12.436984+0000","last_clean":"2024-08-25T21:08:12.436984+0000","last_became_active":"2024-08-25T19:46:21.318722+0000","last_became_peered":"2024-08-25T19:46:21.318722+0000","last_unstale":"2024-08-25T21:08:12.436984+0000","last_undegraded":"2024-08-25T21:08:12.436984+0000","last_fullsized":"2024-08-25T21:08:12.436984+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"1924'2","last_scrub_stamp":"2024-08-25T21:08:12.436889+0000","last_deep_scrub":"1924'2","last_deep_scrub_stamp":"2024-08-25T21:08:12.436889+0000","last_clean_scrub_stamp":"2024-08-25T21:08:12.436889+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:51:35.311674+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00099496299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"1924'54","reported_seq":3956,"reported_epoch":1924,"state":"active+clean","last_fresh":"2024-08-25T21:08:11.440681+0000","last_change":"2024-08-25T21:08:11.440711+0000","last_active":"2024-08-25T21:08:11.440681+0000","last_peered":"2024-08-25T21:08:11.440681+0000","last_clean":"2024-08-25T21:08:11.440681+0000","last_became_active":"2024-08-25T19:46:17.304344+0000","last_became_peered":"2024-08-25T19:46:17.304344+0000","last_unstale":"2024-08-25T21:08:11.440681+0000","last_undegraded":"2024-08-25T21:08:11.440681+0000","last_fullsized":"2024-08-25T21:08:11.440681+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"1924'54","last_scrub_stamp":"2024-08-25T21:08:11.440612+0000","last_deep_scrub":"1924'54","last_deep_scrub_stamp":"2024-08-25T21:08:11.440612+0000","last_clean_scrub_stamp":"2024-08-25T21:08:11.440612+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:52:39.891505+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739358,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708176,"kb_used_data":5056,"kb_used_omap":1,"kb_used_meta":703102,"kb_avail":93663664,"statfs":{"total":96636764160,"available":95911591936,"internally_reserved":0,"allocated":5177344,"data_stored":2849004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":719976901},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739359,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803160,"kb_used_data":5640,"kb_used_omap":1,"kb_used_meta":797502,"kb_avail":93568680,"statfs":{"total":96636764160,"available":95814328320,"internally_reserved":0,"allocated":5775360,"data_stored":3439391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":816642501},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739358,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907288,"kb_used_data":5640,"kb_used_omap":1,"kb_used_meta":901630,"kb_avail":93464552,"statfs":{"total":96636764160,"available":95707701248,"internally_reserved":0,"allocated":5775360,"data_stored":3439391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":923269574},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T21:08:31.855 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster cluster2 config set global mon_health_to_clog false 2024-08-25T21:08:32.222 INFO:teuthology.misc:Shutting down mds daemons... 2024-08-25T21:08:32.223 INFO:teuthology.misc:Shutting down osd daemons... 2024-08-25T21:08:32.223 DEBUG:tasks.ceph.cluster2.osd.0:waiting for process to exit 2024-08-25T21:08:32.223 INFO:teuthology.orchestra.run:waiting for 300 2024-08-25T21:08:35.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.0 is failed for ~0s 2024-08-25T21:08:38.324 INFO:tasks.ceph.cluster2.osd.0:Stopped 2024-08-25T21:08:38.325 DEBUG:tasks.ceph.cluster2.osd.1:waiting for process to exit 2024-08-25T21:08:38.325 INFO:teuthology.orchestra.run:waiting for 300 2024-08-25T21:08:41.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.1 is failed for ~0s 2024-08-25T21:08:41.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.0 has been restored 2024-08-25T21:08:44.426 INFO:tasks.ceph.cluster2.osd.1:Stopped 2024-08-25T21:08:44.427 DEBUG:tasks.ceph.cluster2.osd.2:waiting for process to exit 2024-08-25T21:08:44.427 INFO:teuthology.orchestra.run:waiting for 300 2024-08-25T21:08:46.423 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.2 is failed for ~0s 2024-08-25T21:08:46.423 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.1 has been restored 2024-08-25T21:08:50.528 INFO:tasks.ceph.cluster2.osd.2:Stopped 2024-08-25T21:08:50.529 INFO:teuthology.misc:Shutting down mgr daemons... 2024-08-25T21:08:50.529 DEBUG:tasks.ceph.cluster2.mgr.x:waiting for process to exit 2024-08-25T21:08:50.529 INFO:teuthology.orchestra.run:waiting for 300 2024-08-25T21:08:50.614 INFO:tasks.ceph.cluster2.mgr.x:Stopped 2024-08-25T21:08:50.614 INFO:teuthology.misc:Shutting down mon daemons... 2024-08-25T21:08:50.614 DEBUG:tasks.ceph.cluster2.mon.a:waiting for process to exit 2024-08-25T21:08:50.614 INFO:teuthology.orchestra.run:waiting for 300 2024-08-25T21:08:50.653 INFO:tasks.ceph.cluster2.mon.a:Stopped 2024-08-25T21:08:50.653 INFO:tasks.ceph:Checking cluster log for badness... 2024-08-25T21:08:50.653 DEBUG:teuthology.orchestra.run.smithi121:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/cluster2.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-08-25T21:08:50.800 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster2-0 on ubuntu@smithi121.front.sepia.ceph.com 2024-08-25T21:08:50.800 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/cluster2-0 2024-08-25T21:08:51.050 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster2-1 on ubuntu@smithi121.front.sepia.ceph.com 2024-08-25T21:08:51.050 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/cluster2-1 2024-08-25T21:08:51.183 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster2-2 on ubuntu@smithi121.front.sepia.ceph.com 2024-08-25T21:08:51.183 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/cluster2-2 2024-08-25T21:08:51.315 INFO:tasks.ceph:Archiving mon data... 2024-08-25T21:08:51.316 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/mon/cluster2-a to /home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544/data/cluster2.mon.a.tgz 2024-08-25T21:08:51.316 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-08-25T21:08:51.357 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.V2QNRpaHT0 2024-08-25T21:08:51.358 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar cz -f - -C /var/lib/ceph/mon/cluster2-a -- . > /tmp/tmp.V2QNRpaHT0 2024-08-25T21:08:51.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.2 has been restored 2024-08-25T21:08:51.583 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.V2QNRpaHT0 2024-08-25T21:08:51.700 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.V2QNRpaHT0 is 509KB 2024-08-25T21:08:51.793 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.V2QNRpaHT0 2024-08-25T21:08:51.827 INFO:tasks.ceph:Cleaning ceph cluster... 2024-08-25T21:08:51.828 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /etc/ceph/cluster2.conf /etc/ceph/cluster2.keyring /home/ubuntu/cephtest/cluster2.data /home/ubuntu/cephtest/cluster2.monmap /home/ubuntu/cephtest/../*.pid 2024-08-25T21:08:51.833 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /etc/ceph/cluster2.conf /etc/ceph/cluster2.keyring /home/ubuntu/cephtest/cluster2.data /home/ubuntu/cephtest/cluster2.monmap /home/ubuntu/cephtest/../*.pid 2024-08-25T21:08:52.275 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-08-25T21:08:52.287 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2024-08-25T21:08:52.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-08-25T21:08:52.897 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T21:08:52.898 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-08-25T21:08:52.917 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":4404,"stamp":"2024-08-25T21:08:52.283253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3879524,"kb_used_data":15388,"kb_used_omap":4,"kb_used_meta":3864059,"kb_avail":279235996,"statfs":{"total":289910292480,"available":285937659904,"internally_reserved":0,"allocated":15757312,"data_stored":8704252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":3956796751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002138"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3846,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.040297+0000","last_change":"2024-08-25T19:45:45.028870+0000","last_active":"2024-08-25T21:08:04.040297+0000","last_peered":"2024-08-25T21:08:04.040297+0000","last_clean":"2024-08-25T21:08:04.040297+0000","last_became_active":"2024-08-25T19:45:43.023302+0000","last_became_peered":"2024-08-25T19:45:43.023302+0000","last_unstale":"2024-08-25T21:08:04.040297+0000","last_undegraded":"2024-08-25T21:08:04.040297+0000","last_fullsized":"2024-08-25T21:08:04.040297+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:01:59.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001138303,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3846,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.040367+0000","last_change":"2024-08-25T19:45:45.028517+0000","last_active":"2024-08-25T21:08:04.040367+0000","last_peered":"2024-08-25T21:08:04.040367+0000","last_clean":"2024-08-25T21:08:04.040367+0000","last_became_active":"2024-08-25T19:45:43.022938+0000","last_became_peered":"2024-08-25T19:45:43.022938+0000","last_unstale":"2024-08-25T21:08:04.040367+0000","last_undegraded":"2024-08-25T21:08:04.040367+0000","last_fullsized":"2024-08-25T21:08:04.040367+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:07:13.362353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081442199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3847,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.036539+0000","last_change":"2024-08-25T19:45:45.028575+0000","last_active":"2024-08-25T21:08:04.036539+0000","last_peered":"2024-08-25T21:08:04.036539+0000","last_clean":"2024-08-25T21:08:04.036539+0000","last_became_active":"2024-08-25T19:45:43.024909+0000","last_became_peered":"2024-08-25T19:45:43.024909+0000","last_unstale":"2024-08-25T21:08:04.036539+0000","last_undegraded":"2024-08-25T21:08:04.036539+0000","last_fullsized":"2024-08-25T21:08:04.036539+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T04:09:46.949509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089386499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3847,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.036498+0000","last_change":"2024-08-25T19:45:45.028879+0000","last_active":"2024-08-25T21:08:04.036498+0000","last_peered":"2024-08-25T21:08:04.036498+0000","last_clean":"2024-08-25T21:08:04.036498+0000","last_became_active":"2024-08-25T19:45:43.022595+0000","last_became_peered":"2024-08-25T19:45:43.022595+0000","last_unstale":"2024-08-25T21:08:04.036498+0000","last_undegraded":"2024-08-25T21:08:04.036498+0000","last_fullsized":"2024-08-25T21:08:04.036498+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:12:22.899460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159848,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":3849,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.330693+0000","last_change":"2024-08-25T19:45:45.023825+0000","last_active":"2024-08-25T21:08:04.330693+0000","last_peered":"2024-08-25T21:08:04.330693+0000","last_clean":"2024-08-25T21:08:04.330693+0000","last_became_active":"2024-08-25T19:45:43.024429+0000","last_became_peered":"2024-08-25T19:45:43.024429+0000","last_unstale":"2024-08-25T21:08:04.330693+0000","last_undegraded":"2024-08-25T21:08:04.330693+0000","last_fullsized":"2024-08-25T21:08:04.330693+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:32:06.467397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051996799999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3845,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.329891+0000","last_change":"2024-08-25T19:45:45.095179+0000","last_active":"2024-08-25T21:08:04.329891+0000","last_peered":"2024-08-25T21:08:04.329891+0000","last_clean":"2024-08-25T21:08:04.329891+0000","last_became_active":"2024-08-25T19:45:43.023906+0000","last_became_peered":"2024-08-25T19:45:43.023906+0000","last_unstale":"2024-08-25T21:08:04.329891+0000","last_undegraded":"2024-08-25T21:08:04.329891+0000","last_fullsized":"2024-08-25T21:08:04.329891+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T02:06:50.439446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041333800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3847,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.324422+0000","last_change":"2024-08-25T19:45:45.095342+0000","last_active":"2024-08-25T21:08:04.324422+0000","last_peered":"2024-08-25T21:08:04.324422+0000","last_clean":"2024-08-25T21:08:04.324422+0000","last_became_active":"2024-08-25T19:45:43.023267+0000","last_became_peered":"2024-08-25T19:45:43.023267+0000","last_unstale":"2024-08-25T21:08:04.324422+0000","last_undegraded":"2024-08-25T21:08:04.324422+0000","last_fullsized":"2024-08-25T21:08:04.324422+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:32:06.467397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055963200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":3848,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.037471+0000","last_change":"2024-08-25T19:45:45.029359+0000","last_active":"2024-08-25T21:08:04.037471+0000","last_peered":"2024-08-25T21:08:04.037471+0000","last_clean":"2024-08-25T21:08:04.037471+0000","last_became_active":"2024-08-25T19:45:43.022303+0000","last_became_peered":"2024-08-25T19:45:43.022303+0000","last_unstale":"2024-08-25T21:08:04.037471+0000","last_undegraded":"2024-08-25T21:08:04.037471+0000","last_fullsized":"2024-08-25T21:08:04.037471+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:45:54.687035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637349,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":3917,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.035598+0000","last_change":"2024-08-25T19:45:39.001678+0000","last_active":"2024-08-25T21:08:04.035598+0000","last_peered":"2024-08-25T21:08:04.035598+0000","last_clean":"2024-08-25T21:08:04.035598+0000","last_became_active":"2024-08-25T19:45:38.999237+0000","last_became_peered":"2024-08-25T19:45:38.999237+0000","last_unstale":"2024-08-25T21:08:04.035598+0000","last_undegraded":"2024-08-25T21:08:04.035598+0000","last_fullsized":"2024-08-25T21:08:04.035598+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:37.987775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:37.987775+0000","last_clean_scrub_stamp":"2024-08-25T19:45:37.987775+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:49:16.221195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739370,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1217308,"kb_used_data":4740,"kb_used_omap":1,"kb_used_meta":1212542,"kb_avail":93154532,"statfs":{"total":96636764160,"available":95390240768,"internally_reserved":0,"allocated":4853760,"data_stored":2507826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1241643461},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739370,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1326244,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":1320894,"kb_avail":93045596,"statfs":{"total":96636764160,"available":95278690304,"internally_reserved":0,"allocated":5451776,"data_stored":3098213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1352595909},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1335972,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":1330622,"kb_avail":93035868,"statfs":{"total":96636764160,"available":95268728832,"internally_reserved":0,"allocated":5451776,"data_stored":3098213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1362557381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T21:08:52.917 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-08-25T21:08:53.276 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T21:08:53.276 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-08-25T21:08:53.293 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":4404,"stamp":"2024-08-25T21:08:52.283253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3879524,"kb_used_data":15388,"kb_used_omap":4,"kb_used_meta":3864059,"kb_avail":279235996,"statfs":{"total":289910292480,"available":285937659904,"internally_reserved":0,"allocated":15757312,"data_stored":8704252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":3956796751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002138"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3846,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.040297+0000","last_change":"2024-08-25T19:45:45.028870+0000","last_active":"2024-08-25T21:08:04.040297+0000","last_peered":"2024-08-25T21:08:04.040297+0000","last_clean":"2024-08-25T21:08:04.040297+0000","last_became_active":"2024-08-25T19:45:43.023302+0000","last_became_peered":"2024-08-25T19:45:43.023302+0000","last_unstale":"2024-08-25T21:08:04.040297+0000","last_undegraded":"2024-08-25T21:08:04.040297+0000","last_fullsized":"2024-08-25T21:08:04.040297+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:01:59.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001138303,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3846,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.040367+0000","last_change":"2024-08-25T19:45:45.028517+0000","last_active":"2024-08-25T21:08:04.040367+0000","last_peered":"2024-08-25T21:08:04.040367+0000","last_clean":"2024-08-25T21:08:04.040367+0000","last_became_active":"2024-08-25T19:45:43.022938+0000","last_became_peered":"2024-08-25T19:45:43.022938+0000","last_unstale":"2024-08-25T21:08:04.040367+0000","last_undegraded":"2024-08-25T21:08:04.040367+0000","last_fullsized":"2024-08-25T21:08:04.040367+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:07:13.362353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081442199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3847,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.036539+0000","last_change":"2024-08-25T19:45:45.028575+0000","last_active":"2024-08-25T21:08:04.036539+0000","last_peered":"2024-08-25T21:08:04.036539+0000","last_clean":"2024-08-25T21:08:04.036539+0000","last_became_active":"2024-08-25T19:45:43.024909+0000","last_became_peered":"2024-08-25T19:45:43.024909+0000","last_unstale":"2024-08-25T21:08:04.036539+0000","last_undegraded":"2024-08-25T21:08:04.036539+0000","last_fullsized":"2024-08-25T21:08:04.036539+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T04:09:46.949509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089386499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3847,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.036498+0000","last_change":"2024-08-25T19:45:45.028879+0000","last_active":"2024-08-25T21:08:04.036498+0000","last_peered":"2024-08-25T21:08:04.036498+0000","last_clean":"2024-08-25T21:08:04.036498+0000","last_became_active":"2024-08-25T19:45:43.022595+0000","last_became_peered":"2024-08-25T19:45:43.022595+0000","last_unstale":"2024-08-25T21:08:04.036498+0000","last_undegraded":"2024-08-25T21:08:04.036498+0000","last_fullsized":"2024-08-25T21:08:04.036498+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:12:22.899460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159848,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":3849,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.330693+0000","last_change":"2024-08-25T19:45:45.023825+0000","last_active":"2024-08-25T21:08:04.330693+0000","last_peered":"2024-08-25T21:08:04.330693+0000","last_clean":"2024-08-25T21:08:04.330693+0000","last_became_active":"2024-08-25T19:45:43.024429+0000","last_became_peered":"2024-08-25T19:45:43.024429+0000","last_unstale":"2024-08-25T21:08:04.330693+0000","last_undegraded":"2024-08-25T21:08:04.330693+0000","last_fullsized":"2024-08-25T21:08:04.330693+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:32:06.467397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051996799999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3845,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.329891+0000","last_change":"2024-08-25T19:45:45.095179+0000","last_active":"2024-08-25T21:08:04.329891+0000","last_peered":"2024-08-25T21:08:04.329891+0000","last_clean":"2024-08-25T21:08:04.329891+0000","last_became_active":"2024-08-25T19:45:43.023906+0000","last_became_peered":"2024-08-25T19:45:43.023906+0000","last_unstale":"2024-08-25T21:08:04.329891+0000","last_undegraded":"2024-08-25T21:08:04.329891+0000","last_fullsized":"2024-08-25T21:08:04.329891+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T02:06:50.439446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041333800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3847,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.324422+0000","last_change":"2024-08-25T19:45:45.095342+0000","last_active":"2024-08-25T21:08:04.324422+0000","last_peered":"2024-08-25T21:08:04.324422+0000","last_clean":"2024-08-25T21:08:04.324422+0000","last_became_active":"2024-08-25T19:45:43.023267+0000","last_became_peered":"2024-08-25T19:45:43.023267+0000","last_unstale":"2024-08-25T21:08:04.324422+0000","last_undegraded":"2024-08-25T21:08:04.324422+0000","last_fullsized":"2024-08-25T21:08:04.324422+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:32:06.467397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055963200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":3848,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.037471+0000","last_change":"2024-08-25T19:45:45.029359+0000","last_active":"2024-08-25T21:08:04.037471+0000","last_peered":"2024-08-25T21:08:04.037471+0000","last_clean":"2024-08-25T21:08:04.037471+0000","last_became_active":"2024-08-25T19:45:43.022303+0000","last_became_peered":"2024-08-25T19:45:43.022303+0000","last_unstale":"2024-08-25T21:08:04.037471+0000","last_undegraded":"2024-08-25T21:08:04.037471+0000","last_fullsized":"2024-08-25T21:08:04.037471+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:45:54.687035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637349,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":3917,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.035598+0000","last_change":"2024-08-25T19:45:39.001678+0000","last_active":"2024-08-25T21:08:04.035598+0000","last_peered":"2024-08-25T21:08:04.035598+0000","last_clean":"2024-08-25T21:08:04.035598+0000","last_became_active":"2024-08-25T19:45:38.999237+0000","last_became_peered":"2024-08-25T19:45:38.999237+0000","last_unstale":"2024-08-25T21:08:04.035598+0000","last_undegraded":"2024-08-25T21:08:04.035598+0000","last_fullsized":"2024-08-25T21:08:04.035598+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:37.987775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:37.987775+0000","last_clean_scrub_stamp":"2024-08-25T19:45:37.987775+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:49:16.221195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739370,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1217308,"kb_used_data":4740,"kb_used_omap":1,"kb_used_meta":1212542,"kb_avail":93154532,"statfs":{"total":96636764160,"available":95390240768,"internally_reserved":0,"allocated":4853760,"data_stored":2507826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1241643461},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739370,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1326244,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":1320894,"kb_avail":93045596,"statfs":{"total":96636764160,"available":95278690304,"internally_reserved":0,"allocated":5451776,"data_stored":3098213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1352595909},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1335972,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":1330622,"kb_avail":93035868,"statfs":{"total":96636764160,"available":95268728832,"internally_reserved":0,"allocated":5451776,"data_stored":3098213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1362557381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T21:08:53.294 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2024-08-25T21:08:53.294 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-08-25T21:08:53.651 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T21:08:53.652 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-08-25T21:08:53.669 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":4404,"stamp":"2024-08-25T21:08:52.283253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3879524,"kb_used_data":15388,"kb_used_omap":4,"kb_used_meta":3864059,"kb_avail":279235996,"statfs":{"total":289910292480,"available":285937659904,"internally_reserved":0,"allocated":15757312,"data_stored":8704252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":3956796751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002138"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3846,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.040297+0000","last_change":"2024-08-25T19:45:45.028870+0000","last_active":"2024-08-25T21:08:04.040297+0000","last_peered":"2024-08-25T21:08:04.040297+0000","last_clean":"2024-08-25T21:08:04.040297+0000","last_became_active":"2024-08-25T19:45:43.023302+0000","last_became_peered":"2024-08-25T19:45:43.023302+0000","last_unstale":"2024-08-25T21:08:04.040297+0000","last_undegraded":"2024-08-25T21:08:04.040297+0000","last_fullsized":"2024-08-25T21:08:04.040297+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:01:59.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001138303,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3846,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.040367+0000","last_change":"2024-08-25T19:45:45.028517+0000","last_active":"2024-08-25T21:08:04.040367+0000","last_peered":"2024-08-25T21:08:04.040367+0000","last_clean":"2024-08-25T21:08:04.040367+0000","last_became_active":"2024-08-25T19:45:43.022938+0000","last_became_peered":"2024-08-25T19:45:43.022938+0000","last_unstale":"2024-08-25T21:08:04.040367+0000","last_undegraded":"2024-08-25T21:08:04.040367+0000","last_fullsized":"2024-08-25T21:08:04.040367+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:07:13.362353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081442199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3847,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.036539+0000","last_change":"2024-08-25T19:45:45.028575+0000","last_active":"2024-08-25T21:08:04.036539+0000","last_peered":"2024-08-25T21:08:04.036539+0000","last_clean":"2024-08-25T21:08:04.036539+0000","last_became_active":"2024-08-25T19:45:43.024909+0000","last_became_peered":"2024-08-25T19:45:43.024909+0000","last_unstale":"2024-08-25T21:08:04.036539+0000","last_undegraded":"2024-08-25T21:08:04.036539+0000","last_fullsized":"2024-08-25T21:08:04.036539+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T04:09:46.949509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089386499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3847,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.036498+0000","last_change":"2024-08-25T19:45:45.028879+0000","last_active":"2024-08-25T21:08:04.036498+0000","last_peered":"2024-08-25T21:08:04.036498+0000","last_clean":"2024-08-25T21:08:04.036498+0000","last_became_active":"2024-08-25T19:45:43.022595+0000","last_became_peered":"2024-08-25T19:45:43.022595+0000","last_unstale":"2024-08-25T21:08:04.036498+0000","last_undegraded":"2024-08-25T21:08:04.036498+0000","last_fullsized":"2024-08-25T21:08:04.036498+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:12:22.899460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159848,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":3849,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.330693+0000","last_change":"2024-08-25T19:45:45.023825+0000","last_active":"2024-08-25T21:08:04.330693+0000","last_peered":"2024-08-25T21:08:04.330693+0000","last_clean":"2024-08-25T21:08:04.330693+0000","last_became_active":"2024-08-25T19:45:43.024429+0000","last_became_peered":"2024-08-25T19:45:43.024429+0000","last_unstale":"2024-08-25T21:08:04.330693+0000","last_undegraded":"2024-08-25T21:08:04.330693+0000","last_fullsized":"2024-08-25T21:08:04.330693+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:32:06.467397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051996799999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3845,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.329891+0000","last_change":"2024-08-25T19:45:45.095179+0000","last_active":"2024-08-25T21:08:04.329891+0000","last_peered":"2024-08-25T21:08:04.329891+0000","last_clean":"2024-08-25T21:08:04.329891+0000","last_became_active":"2024-08-25T19:45:43.023906+0000","last_became_peered":"2024-08-25T19:45:43.023906+0000","last_unstale":"2024-08-25T21:08:04.329891+0000","last_undegraded":"2024-08-25T21:08:04.329891+0000","last_fullsized":"2024-08-25T21:08:04.329891+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T02:06:50.439446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041333800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3847,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.324422+0000","last_change":"2024-08-25T19:45:45.095342+0000","last_active":"2024-08-25T21:08:04.324422+0000","last_peered":"2024-08-25T21:08:04.324422+0000","last_clean":"2024-08-25T21:08:04.324422+0000","last_became_active":"2024-08-25T19:45:43.023267+0000","last_became_peered":"2024-08-25T19:45:43.023267+0000","last_unstale":"2024-08-25T21:08:04.324422+0000","last_undegraded":"2024-08-25T21:08:04.324422+0000","last_fullsized":"2024-08-25T21:08:04.324422+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:32:06.467397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055963200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":3848,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.037471+0000","last_change":"2024-08-25T19:45:45.029359+0000","last_active":"2024-08-25T21:08:04.037471+0000","last_peered":"2024-08-25T21:08:04.037471+0000","last_clean":"2024-08-25T21:08:04.037471+0000","last_became_active":"2024-08-25T19:45:43.022303+0000","last_became_peered":"2024-08-25T19:45:43.022303+0000","last_unstale":"2024-08-25T21:08:04.037471+0000","last_undegraded":"2024-08-25T21:08:04.037471+0000","last_fullsized":"2024-08-25T21:08:04.037471+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:45:54.687035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637349,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":3917,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.035598+0000","last_change":"2024-08-25T19:45:39.001678+0000","last_active":"2024-08-25T21:08:04.035598+0000","last_peered":"2024-08-25T21:08:04.035598+0000","last_clean":"2024-08-25T21:08:04.035598+0000","last_became_active":"2024-08-25T19:45:38.999237+0000","last_became_peered":"2024-08-25T19:45:38.999237+0000","last_unstale":"2024-08-25T21:08:04.035598+0000","last_undegraded":"2024-08-25T21:08:04.035598+0000","last_fullsized":"2024-08-25T21:08:04.035598+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:37.987775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:37.987775+0000","last_clean_scrub_stamp":"2024-08-25T19:45:37.987775+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:49:16.221195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739370,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1217308,"kb_used_data":4740,"kb_used_omap":1,"kb_used_meta":1212542,"kb_avail":93154532,"statfs":{"total":96636764160,"available":95390240768,"internally_reserved":0,"allocated":4853760,"data_stored":2507826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1241643461},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739370,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1326244,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":1320894,"kb_avail":93045596,"statfs":{"total":96636764160,"available":95278690304,"internally_reserved":0,"allocated":5451776,"data_stored":3098213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1352595909},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1335972,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":1330622,"kb_avail":93035868,"statfs":{"total":96636764160,"available":95268728832,"internally_reserved":0,"allocated":5451776,"data_stored":3098213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1362557381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T21:08:53.670 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2024-08-25T21:08:54.030 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T21:08:54.030 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":1909,"fsid":"c5c74996-f4b7-4137-83c8-02707cfa2a6b","created":"2024-08-25T19:45:25.903665+0000","modified":"2024-08-25T21:08:04.028288+0000","last_up_change":"2024-08-25T19:45:36.760528+0000","last_in_change":"2024-08-25T19:45:27.025216+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-25T19:45:36.999898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-08-25T19:45:41.639916+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac2735a0-67f3-4768-a027-9f903902b077","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6801","nonce":3716486698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6803","nonce":3716486698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6807","nonce":3716486698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3716486698},{"type":"v1","addr":"172.21.15.119:6805","nonce":3716486698}]},"public_addr":"172.21.15.119:6801/3716486698","cluster_addr":"172.21.15.119:6803/3716486698","heartbeat_back_addr":"172.21.15.119:6807/3716486698","heartbeat_front_addr":"172.21.15.119:6805/3716486698","state":["exists","up"]},{"osd":1,"uuid":"e9fa7c61-90e7-415e-a1cd-f68cb71ac72f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6809","nonce":1091288145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6811","nonce":1091288145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6815","nonce":1091288145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":1091288145},{"type":"v1","addr":"172.21.15.119:6813","nonce":1091288145}]},"public_addr":"172.21.15.119:6809/1091288145","cluster_addr":"172.21.15.119:6811/1091288145","heartbeat_back_addr":"172.21.15.119:6815/1091288145","heartbeat_front_addr":"172.21.15.119:6813/1091288145","state":["exists","up"]},{"osd":2,"uuid":"234bf606-0a17-42e0-a74a-de1639d9e84d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6817","nonce":944614458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6819","nonce":944614458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6823","nonce":944614458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":944614458},{"type":"v1","addr":"172.21.15.119:6821","nonce":944614458}]},"public_addr":"172.21.15.119:6817/944614458","cluster_addr":"172.21.15.119:6819/944614458","heartbeat_back_addr":"172.21.15.119:6823/944614458","heartbeat_front_addr":"172.21.15.119:6821/944614458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:45:33.257695+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:45:33.296871+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-25T19:45:33.527084+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":2979,"length":2},{"begin":2989,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-25T21:08:55.049 INFO:tasks.ceph:Scrubbing cluster1.osd.0 2024-08-25T21:08:55.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-08-25T21:08:55.231 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-08-25T21:08:55.232 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-25T21:08:55.232 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-08-25T21:08:55.247 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd deep-scrub 0 2024-08-25T21:08:55.595 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 0 to deep-scrub 2024-08-25T21:08:55.614 INFO:tasks.ceph:Scrubbing cluster1.osd.1 2024-08-25T21:08:55.614 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-08-25T21:08:55.792 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-08-25T21:08:55.792 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-25T21:08:55.792 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-08-25T21:08:55.807 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd deep-scrub 1 2024-08-25T21:08:56.156 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 1 to deep-scrub 2024-08-25T21:08:56.176 INFO:tasks.ceph:Scrubbing cluster1.osd.2 2024-08-25T21:08:56.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-08-25T21:08:56.353 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-08-25T21:08:56.354 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-25T21:08:56.354 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-08-25T21:08:56.368 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd deep-scrub 2 2024-08-25T21:08:56.716 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 2 to deep-scrub 2024-08-25T21:08:56.734 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-08-25T21:08:57.083 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T21:08:57.083 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-08-25T21:08:57.101 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":4406,"stamp":"2024-08-25T21:08:56.283892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3879524,"kb_used_data":15388,"kb_used_omap":4,"kb_used_meta":3864059,"kb_avail":279235996,"statfs":{"total":289910292480,"available":285937659904,"internally_reserved":0,"allocated":15757312,"data_stored":8704252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":3956796751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002087"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3846,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.040297+0000","last_change":"2024-08-25T19:45:45.028870+0000","last_active":"2024-08-25T21:08:04.040297+0000","last_peered":"2024-08-25T21:08:04.040297+0000","last_clean":"2024-08-25T21:08:04.040297+0000","last_became_active":"2024-08-25T19:45:43.023302+0000","last_became_peered":"2024-08-25T19:45:43.023302+0000","last_unstale":"2024-08-25T21:08:04.040297+0000","last_undegraded":"2024-08-25T21:08:04.040297+0000","last_fullsized":"2024-08-25T21:08:04.040297+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:01:59.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001138303,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3846,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.040367+0000","last_change":"2024-08-25T19:45:45.028517+0000","last_active":"2024-08-25T21:08:04.040367+0000","last_peered":"2024-08-25T21:08:04.040367+0000","last_clean":"2024-08-25T21:08:04.040367+0000","last_became_active":"2024-08-25T19:45:43.022938+0000","last_became_peered":"2024-08-25T19:45:43.022938+0000","last_unstale":"2024-08-25T21:08:04.040367+0000","last_undegraded":"2024-08-25T21:08:04.040367+0000","last_fullsized":"2024-08-25T21:08:04.040367+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:07:13.362353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081442199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3847,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.036539+0000","last_change":"2024-08-25T19:45:45.028575+0000","last_active":"2024-08-25T21:08:04.036539+0000","last_peered":"2024-08-25T21:08:04.036539+0000","last_clean":"2024-08-25T21:08:04.036539+0000","last_became_active":"2024-08-25T19:45:43.024909+0000","last_became_peered":"2024-08-25T19:45:43.024909+0000","last_unstale":"2024-08-25T21:08:04.036539+0000","last_undegraded":"2024-08-25T21:08:04.036539+0000","last_fullsized":"2024-08-25T21:08:04.036539+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T04:09:46.949509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089386499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3847,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.036498+0000","last_change":"2024-08-25T19:45:45.028879+0000","last_active":"2024-08-25T21:08:04.036498+0000","last_peered":"2024-08-25T21:08:04.036498+0000","last_clean":"2024-08-25T21:08:04.036498+0000","last_became_active":"2024-08-25T19:45:43.022595+0000","last_became_peered":"2024-08-25T19:45:43.022595+0000","last_unstale":"2024-08-25T21:08:04.036498+0000","last_undegraded":"2024-08-25T21:08:04.036498+0000","last_fullsized":"2024-08-25T21:08:04.036498+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-26T21:12:22.899460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159848,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":3849,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.330693+0000","last_change":"2024-08-25T19:45:45.023825+0000","last_active":"2024-08-25T21:08:04.330693+0000","last_peered":"2024-08-25T21:08:04.330693+0000","last_clean":"2024-08-25T21:08:04.330693+0000","last_became_active":"2024-08-25T19:45:43.024429+0000","last_became_peered":"2024-08-25T19:45:43.024429+0000","last_unstale":"2024-08-25T21:08:04.330693+0000","last_undegraded":"2024-08-25T21:08:04.330693+0000","last_fullsized":"2024-08-25T21:08:04.330693+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:32:06.467397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051996799999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3845,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.329891+0000","last_change":"2024-08-25T19:45:45.095179+0000","last_active":"2024-08-25T21:08:04.329891+0000","last_peered":"2024-08-25T21:08:04.329891+0000","last_clean":"2024-08-25T21:08:04.329891+0000","last_became_active":"2024-08-25T19:45:43.023906+0000","last_became_peered":"2024-08-25T19:45:43.023906+0000","last_unstale":"2024-08-25T21:08:04.329891+0000","last_undegraded":"2024-08-25T21:08:04.329891+0000","last_fullsized":"2024-08-25T21:08:04.329891+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T02:06:50.439446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041333800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3847,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.324422+0000","last_change":"2024-08-25T19:45:45.095342+0000","last_active":"2024-08-25T21:08:04.324422+0000","last_peered":"2024-08-25T21:08:04.324422+0000","last_clean":"2024-08-25T21:08:04.324422+0000","last_became_active":"2024-08-25T19:45:43.023267+0000","last_became_peered":"2024-08-25T19:45:43.023267+0000","last_unstale":"2024-08-25T21:08:04.324422+0000","last_undegraded":"2024-08-25T21:08:04.324422+0000","last_fullsized":"2024-08-25T21:08:04.324422+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:32:06.467397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055963200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":3848,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.037471+0000","last_change":"2024-08-25T19:45:45.029359+0000","last_active":"2024-08-25T21:08:04.037471+0000","last_peered":"2024-08-25T21:08:04.037471+0000","last_clean":"2024-08-25T21:08:04.037471+0000","last_became_active":"2024-08-25T19:45:43.022303+0000","last_became_peered":"2024-08-25T19:45:43.022303+0000","last_unstale":"2024-08-25T21:08:04.037471+0000","last_undegraded":"2024-08-25T21:08:04.037471+0000","last_fullsized":"2024-08-25T21:08:04.037471+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:42.004110+0000","last_clean_scrub_stamp":"2024-08-25T19:45:42.004110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:45:54.687035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637349,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":3917,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:04.035598+0000","last_change":"2024-08-25T19:45:39.001678+0000","last_active":"2024-08-25T21:08:04.035598+0000","last_peered":"2024-08-25T21:08:04.035598+0000","last_clean":"2024-08-25T21:08:04.035598+0000","last_became_active":"2024-08-25T19:45:38.999237+0000","last_became_peered":"2024-08-25T19:45:38.999237+0000","last_unstale":"2024-08-25T21:08:04.035598+0000","last_undegraded":"2024-08-25T21:08:04.035598+0000","last_fullsized":"2024-08-25T21:08:04.035598+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T19:45:37.987775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T19:45:37.987775+0000","last_clean_scrub_stamp":"2024-08-25T19:45:37.987775+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T05:49:16.221195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739370,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1217308,"kb_used_data":4740,"kb_used_omap":1,"kb_used_meta":1212542,"kb_avail":93154532,"statfs":{"total":96636764160,"available":95390240768,"internally_reserved":0,"allocated":4853760,"data_stored":2507826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1241643461},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739370,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1326244,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":1320894,"kb_avail":93045596,"statfs":{"total":96636764160,"available":95278690304,"internally_reserved":0,"allocated":5451776,"data_stored":3098213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1352595909},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1335972,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":1330622,"kb_avail":93035868,"statfs":{"total":96636764160,"available":95268728832,"internally_reserved":0,"allocated":5451776,"data_stored":3098213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1362557381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T21:08:57.102 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-08-25T19:45:42.004110+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=45, tm_sec=42, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=54, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:57.102 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-08-25T19:45:42.004110+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=45, tm_sec=42, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=54, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:57.102 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-08-25T19:45:42.004110+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=45, tm_sec=42, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=54, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:57.102 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-08-25T19:45:42.004110+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=45, tm_sec=42, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=54, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:57.102 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-08-25T19:45:42.004110+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=45, tm_sec=42, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=54, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:57.102 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-08-25T19:45:42.004110+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=45, tm_sec=42, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=54, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:57.102 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-08-25T19:45:42.004110+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=45, tm_sec=42, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=54, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:57.103 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-08-25T19:45:42.004110+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=45, tm_sec=42, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=54, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:57.103 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-08-25T19:45:37.987775+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=19, tm_min=45, tm_sec=37, tm_wday=6, tm_yday=238, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=25, tm_hour=21, tm_min=8, tm_sec=54, tm_wday=6, tm_yday=238, tm_isdst=0) 2024-08-25T21:08:57.103 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-08-25T21:09:17.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-08-25T21:09:17.458 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-08-25T21:09:17.458 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-08-25T21:09:17.476 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":4416,"stamp":"2024-08-25T21:09:16.288042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59,"ondisk_log_size":59,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3879500,"kb_used_data":15388,"kb_used_omap":4,"kb_used_meta":3864059,"kb_avail":279236020,"statfs":{"total":289910292480,"available":285937684480,"internally_reserved":0,"allocated":15757312,"data_stored":8704252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":3956796751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002422"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3855,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:09:01.326640+0000","last_change":"2024-08-25T21:09:01.326664+0000","last_active":"2024-08-25T21:09:01.326640+0000","last_peered":"2024-08-25T21:09:01.326640+0000","last_clean":"2024-08-25T21:09:01.326640+0000","last_became_active":"2024-08-25T19:45:43.023302+0000","last_became_peered":"2024-08-25T19:45:43.023302+0000","last_unstale":"2024-08-25T21:09:01.326640+0000","last_undegraded":"2024-08-25T21:09:01.326640+0000","last_fullsized":"2024-08-25T21:09:01.326640+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T21:09:01.326540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T21:09:01.326540+0000","last_clean_scrub_stamp":"2024-08-25T21:09:01.326540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:15:34.957341+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001138303,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3855,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:09:00.361378+0000","last_change":"2024-08-25T21:09:00.361408+0000","last_active":"2024-08-25T21:09:00.361378+0000","last_peered":"2024-08-25T21:09:00.361378+0000","last_clean":"2024-08-25T21:09:00.361378+0000","last_became_active":"2024-08-25T19:45:43.022938+0000","last_became_peered":"2024-08-25T19:45:43.022938+0000","last_unstale":"2024-08-25T21:09:00.361378+0000","last_undegraded":"2024-08-25T21:09:00.361378+0000","last_fullsized":"2024-08-25T21:09:00.361378+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T21:09:00.361275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T21:09:00.361275+0000","last_clean_scrub_stamp":"2024-08-25T21:09:00.361275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T06:49:38.695158+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00081442199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3856,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:58.363245+0000","last_change":"2024-08-25T21:08:58.363271+0000","last_active":"2024-08-25T21:08:58.363245+0000","last_peered":"2024-08-25T21:08:58.363245+0000","last_clean":"2024-08-25T21:08:58.363245+0000","last_became_active":"2024-08-25T19:45:43.024909+0000","last_became_peered":"2024-08-25T19:45:43.024909+0000","last_unstale":"2024-08-25T21:08:58.363245+0000","last_undegraded":"2024-08-25T21:08:58.363245+0000","last_fullsized":"2024-08-25T21:08:58.363245+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T21:08:58.363160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T21:08:58.363160+0000","last_clean_scrub_stamp":"2024-08-25T21:08:58.363160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T03:35:02.210915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089386499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3857,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:56.321078+0000","last_change":"2024-08-25T21:08:56.321078+0000","last_active":"2024-08-25T21:08:56.321078+0000","last_peered":"2024-08-25T21:08:56.321078+0000","last_clean":"2024-08-25T21:08:56.321078+0000","last_became_active":"2024-08-25T19:45:43.022595+0000","last_became_peered":"2024-08-25T19:45:43.022595+0000","last_unstale":"2024-08-25T21:08:56.321078+0000","last_undegraded":"2024-08-25T21:08:56.321078+0000","last_fullsized":"2024-08-25T21:08:56.321078+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T21:08:56.320977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T21:08:56.320977+0000","last_clean_scrub_stamp":"2024-08-25T21:08:56.320977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T04:14:14.189036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001159848,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"1909'3","reported_seq":3860,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:56.415231+0000","last_change":"2024-08-25T21:08:56.415231+0000","last_active":"2024-08-25T21:08:56.415231+0000","last_peered":"2024-08-25T21:08:56.415231+0000","last_clean":"2024-08-25T21:08:56.415231+0000","last_became_active":"2024-08-25T19:45:43.024429+0000","last_became_peered":"2024-08-25T19:45:43.024429+0000","last_unstale":"2024-08-25T21:08:56.415231+0000","last_undegraded":"2024-08-25T21:08:56.415231+0000","last_fullsized":"2024-08-25T21:08:56.415231+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"1909'3","last_scrub_stamp":"2024-08-25T21:08:56.415125+0000","last_deep_scrub":"1909'3","last_deep_scrub_stamp":"2024-08-25T21:08:56.415125+0000","last_clean_scrub_stamp":"2024-08-25T21:08:56.415125+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T00:28:33.837330+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00051996799999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3855,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:57.956297+0000","last_change":"2024-08-25T21:08:57.956297+0000","last_active":"2024-08-25T21:08:57.956297+0000","last_peered":"2024-08-25T21:08:57.956297+0000","last_clean":"2024-08-25T21:08:57.956297+0000","last_became_active":"2024-08-25T19:45:43.023906+0000","last_became_peered":"2024-08-25T19:45:43.023906+0000","last_unstale":"2024-08-25T21:08:57.956297+0000","last_undegraded":"2024-08-25T21:08:57.956297+0000","last_fullsized":"2024-08-25T21:08:57.956297+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T21:08:57.956172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T21:08:57.956172+0000","last_clean_scrub_stamp":"2024-08-25T21:08:57.956172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T03:04:37.432852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041333800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3856,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:56.933244+0000","last_change":"2024-08-25T21:08:56.933269+0000","last_active":"2024-08-25T21:08:56.933244+0000","last_peered":"2024-08-25T21:08:56.933244+0000","last_clean":"2024-08-25T21:08:56.933244+0000","last_became_active":"2024-08-25T19:45:43.023267+0000","last_became_peered":"2024-08-25T19:45:43.023267+0000","last_unstale":"2024-08-25T21:08:56.933244+0000","last_undegraded":"2024-08-25T21:08:56.933244+0000","last_fullsized":"2024-08-25T21:08:56.933244+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-25T21:08:56.933165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-25T21:08:56.933165+0000","last_clean_scrub_stamp":"2024-08-25T21:08:56.933165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T07:00:44.809223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055963200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"1909'2","reported_seq":3859,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:57.328872+0000","last_change":"2024-08-25T21:08:57.328872+0000","last_active":"2024-08-25T21:08:57.328872+0000","last_peered":"2024-08-25T21:08:57.328872+0000","last_clean":"2024-08-25T21:08:57.328872+0000","last_became_active":"2024-08-25T19:45:43.022303+0000","last_became_peered":"2024-08-25T19:45:43.022303+0000","last_unstale":"2024-08-25T21:08:57.328872+0000","last_undegraded":"2024-08-25T21:08:57.328872+0000","last_fullsized":"2024-08-25T21:08:57.328872+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"1909'2","last_scrub_stamp":"2024-08-25T21:08:57.328758+0000","last_deep_scrub":"1909'2","last_deep_scrub_stamp":"2024-08-25T21:08:57.328758+0000","last_clean_scrub_stamp":"2024-08-25T21:08:57.328758+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T08:32:52.491234+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001637349,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"1909'54","reported_seq":3928,"reported_epoch":1909,"state":"active+clean","last_fresh":"2024-08-25T21:08:59.394632+0000","last_change":"2024-08-25T21:08:59.394756+0000","last_active":"2024-08-25T21:08:59.394632+0000","last_peered":"2024-08-25T21:08:59.394632+0000","last_clean":"2024-08-25T21:08:59.394632+0000","last_became_active":"2024-08-25T19:45:38.999237+0000","last_became_peered":"2024-08-25T19:45:38.999237+0000","last_unstale":"2024-08-25T21:08:59.394632+0000","last_undegraded":"2024-08-25T21:08:59.394632+0000","last_fullsized":"2024-08-25T21:08:59.394632+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"1909'54","last_scrub_stamp":"2024-08-25T21:08:59.394527+0000","last_deep_scrub":"1909'54","last_deep_scrub_stamp":"2024-08-25T21:08:59.394527+0000","last_clean_scrub_stamp":"2024-08-25T21:08:59.394527+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-27T02:38:14.991405+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739374,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1217300,"kb_used_data":4740,"kb_used_omap":1,"kb_used_meta":1212542,"kb_avail":93154540,"statfs":{"total":96636764160,"available":95390248960,"internally_reserved":0,"allocated":4853760,"data_stored":2507826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1241643461},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739374,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1326236,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":1320894,"kb_avail":93045604,"statfs":{"total":96636764160,"available":95278698496,"internally_reserved":0,"allocated":5451776,"data_stored":3098213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1352595909},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739375,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1335964,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":1330622,"kb_avail":93035876,"statfs":{"total":96636764160,"available":95268737024,"internally_reserved":0,"allocated":5451776,"data_stored":3098213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1362557381},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-25T21:09:17.478 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster cluster1 config set global mon_health_to_clog false 2024-08-25T21:09:17.843 INFO:teuthology.misc:Shutting down mds daemons... 2024-08-25T21:09:17.843 INFO:teuthology.misc:Shutting down osd daemons... 2024-08-25T21:09:17.843 DEBUG:tasks.ceph.cluster1.osd.0:waiting for process to exit 2024-08-25T21:09:17.844 INFO:teuthology.orchestra.run:waiting for 300 2024-08-25T21:09:20.138 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.0 is failed for ~0s 2024-08-25T21:09:23.946 INFO:tasks.ceph.cluster1.osd.0:Stopped 2024-08-25T21:09:23.946 DEBUG:tasks.ceph.cluster1.osd.1:waiting for process to exit 2024-08-25T21:09:23.946 INFO:teuthology.orchestra.run:waiting for 300 2024-08-25T21:09:25.441 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.1 is failed for ~0s 2024-08-25T21:09:25.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.0 has been restored 2024-08-25T21:09:30.048 INFO:tasks.ceph.cluster1.osd.1:Stopped 2024-08-25T21:09:30.048 DEBUG:tasks.ceph.cluster1.osd.2:waiting for process to exit 2024-08-25T21:09:30.048 INFO:teuthology.orchestra.run:waiting for 300 2024-08-25T21:09:30.644 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.2 is failed for ~0s 2024-08-25T21:09:30.644 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.1 has been restored 2024-08-25T21:09:35.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.2 is failed for ~5s 2024-08-25T21:09:36.150 INFO:tasks.ceph.cluster1.osd.2:Stopped 2024-08-25T21:09:36.150 INFO:teuthology.misc:Shutting down mgr daemons... 2024-08-25T21:09:36.151 DEBUG:tasks.ceph.cluster1.mgr.x:waiting for process to exit 2024-08-25T21:09:36.151 INFO:teuthology.orchestra.run:waiting for 300 2024-08-25T21:09:36.230 INFO:tasks.ceph.cluster1.mgr.x:Stopped 2024-08-25T21:09:36.230 INFO:teuthology.misc:Shutting down mon daemons... 2024-08-25T21:09:36.230 DEBUG:tasks.ceph.cluster1.mon.a:waiting for process to exit 2024-08-25T21:09:36.230 INFO:teuthology.orchestra.run:waiting for 300 2024-08-25T21:09:36.265 INFO:tasks.ceph.cluster1.mon.a:Stopped 2024-08-25T21:09:36.265 INFO:tasks.ceph:Checking cluster log for badness... 2024-08-25T21:09:36.266 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/cluster1.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-08-25T21:09:36.423 INFO:teuthology.orchestra.run.smithi119.stdout:2024-08-25T19:48:35.950872+0000 mon.a (mon.0) 206 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-08-25T21:09:36.424 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-08-25T21:09:36.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[SEC\]' /var/log/ceph/cluster1.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-08-25T21:09:36.483 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[ERR\]' /var/log/ceph/cluster1.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-08-25T21:09:36.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[WRN\]' /var/log/ceph/cluster1.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-08-25T21:09:36.599 INFO:teuthology.orchestra.run.smithi119.stdout:2024-08-25T19:48:35.950872+0000 mon.a (mon.0) 206 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs inactive, 3 pgs peering (PG_AVAILABILITY) 2024-08-25T21:09:36.600 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster1-0 on ubuntu@smithi119.front.sepia.ceph.com 2024-08-25T21:09:36.600 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/cluster1-0 2024-08-25T21:09:36.901 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster1-1 on ubuntu@smithi119.front.sepia.ceph.com 2024-08-25T21:09:36.901 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/cluster1-1 2024-08-25T21:09:37.134 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster1-2 on ubuntu@smithi119.front.sepia.ceph.com 2024-08-25T21:09:37.135 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/cluster1-2 2024-08-25T21:09:37.354 INFO:tasks.ceph:Archiving mon data... 2024-08-25T21:09:37.355 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/cluster1-a to /home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544/data/cluster1.mon.a.tgz 2024-08-25T21:09:37.355 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-08-25T21:09:37.390 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.bO96OsTEJJ 2024-08-25T21:09:37.390 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/cluster1-a -- . > /tmp/tmp.bO96OsTEJJ 2024-08-25T21:09:37.585 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.bO96OsTEJJ 2024-08-25T21:09:37.715 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.bO96OsTEJJ is 523KB 2024-08-25T21:09:37.806 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.bO96OsTEJJ 2024-08-25T21:09:37.840 INFO:tasks.ceph:Cleaning ceph cluster... 2024-08-25T21:09:37.840 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /etc/ceph/cluster1.conf /etc/ceph/cluster1.keyring /home/ubuntu/cephtest/cluster1.data /home/ubuntu/cephtest/cluster1.monmap /home/ubuntu/cephtest/../*.pid 2024-08-25T21:09:37.884 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /etc/ceph/cluster1.conf /etc/ceph/cluster1.keyring /home/ubuntu/cephtest/cluster1.data /home/ubuntu/cephtest/cluster1.monmap /home/ubuntu/cephtest/../*.pid 2024-08-25T21:09:37.995 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-08-25T21:09:38.046 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-08-25T21:09:38.047 INFO:tasks.ceph:Archiving crash dumps... 2024-08-25T21:09:38.048 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544/remote/smithi119/crash 2024-08-25T21:09:38.049 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-25T21:09:38.103 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544/remote/smithi121/crash 2024-08-25T21:09:38.104 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-25T21:09:38.159 INFO:tasks.ceph:Compressing logs... 2024-08-25T21:09:38.160 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-25T21:09:38.164 DEBUG:teuthology.orchestra.run.smithi121:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-25T21:09:38.230 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-mgr.x.log 2024-08-25T21:09:38.231 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21752.log 2024-08-25T21:09:38.237 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21625.log 2024-08-25T21:09:38.237 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21752.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21545.log 0.0% 2024-08-25T21:09:38.237 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.21752.log.gz 2024-08-25T21:09:38.237 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21625.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21625.log.gz 2024-08-25T21:09:38.237 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.25917.log 2024-08-25T21:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21545.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21545.log.gz 2024-08-25T21:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.25749.log 2024-08-25T21:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.25917.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.25917.log.gz 2024-08-25T21:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.25833.log 2024-08-25T21:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.25749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.25749.log.gz 2024-08-25T21:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.26001.log 2024-08-25T21:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.25833.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.25833.log.gz 2024-08-25T21:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.12531.log 2024-08-25T21:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.26001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.26001.log.gz 2024-08-25T21:09:38.238 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.25527.log 2024-08-25T21:09:38.240 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.tmp-client.admin.12531.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.12531.log.gz 2024-08-25T21:09:38.240 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.25570.log 2024-08-25T21:09:38.240 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.25527.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.25527.log.gz 2024-08-25T21:09:38.240 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.26041.log 2024-08-25T21:09:38.240 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.25570.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.25570.log.gz 2024-08-25T21:09:38.240 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.17033.log 2024-08-25T21:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.26041.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.26041.log.gz 2024-08-25T21:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.25875.log 2024-08-25T21:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.17033.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.17033.log.gz 2024-08-25T21:09:38.241 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.16835.log 2024-08-25T21:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.25875.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.25875.log.gz 2024-08-25T21:09:38.242 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1.log 2024-08-25T21:09:38.243 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.16835.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.16835.log.gz 2024-08-25T21:09:38.243 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-mon.a.log 2024-08-25T21:09:38.243 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21419.log 2024-08-25T21:09:38.244 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-mon.a.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21368.log 2024-08-25T21:09:38.245 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21419.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21419.log.gz 2024-08-25T21:09:38.245 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-osd.1.log 2024-08-25T21:09:38.246 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21368.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21368.log.gz 2024-08-25T21:09:38.246 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21753.log 2024-08-25T21:09:38.247 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-osd.1.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21503.log 2024-08-25T21:09:38.248 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21753.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21753.log.gz 2024-08-25T21:09:38.248 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21923.log 2024-08-25T21:09:38.248 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21503.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21503.log.gz 2024-08-25T21:09:38.249 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21343.log 2024-08-25T21:09:38.250 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21923.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21923.log.gz 2024-08-25T21:09:38.250 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.20444.log 2024-08-25T21:09:38.250 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21343.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21343.log.gz 2024-08-25T21:09:38.251 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.25791.log 2024-08-25T21:09:38.251 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.20444.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.20444.log.gz 2024-08-25T21:09:38.252 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21722.log 2024-08-25T21:09:38.252 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.25791.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.25791.log.gz 2024-08-25T21:09:38.253 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-osd.0.log 2024-08-25T21:09:38.253 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21722.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21722.log.gz 2024-08-25T21:09:38.253 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21626.log 2024-08-25T21:09:38.254 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/cluster1.log.gz 2024-08-25T21:09:38.254 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-osd.0.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.17240.log 2024-08-25T21:09:38.255 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21626.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21626.log.gz 2024-08-25T21:09:38.255 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.25662.log 2024-08-25T21:09:38.256 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.17240.log: gzip -5 --verbose -- /var/log/ceph/cluster1.audit.log 2024-08-25T21:09:38.256 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.17240.log.gz 2024-08-25T21:09:38.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.25662.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.25662.log.gz 2024-08-25T21:09:38.257 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.17073.log 2024-08-25T21:09:38.258 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1.audit.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21461.log 2024-08-25T21:09:38.259 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.17073.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.17073.log.gz 2024-08-25T21:09:38.259 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.16993.log 2024-08-25T21:09:38.259 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21461.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.25707.log 2024-08-25T21:09:38.259 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21461.log.gz 2024-08-25T21:09:38.260 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.16993.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.16993.log.gz 2024-08-25T21:09:38.260 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.17172.log 2024-08-25T21:09:38.261 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.25707.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.25707.log.gz 2024-08-25T21:09:38.261 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21287.log 2024-08-25T21:09:38.262 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.17172.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.admin.17172.log.gz /var/log/ceph/cluster1-osd.2.log 2024-08-25T21:09:38.262 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-08-25T21:09:38.263 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21287.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21287.log.gz 2024-08-25T21:09:38.263 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21627.log 2024-08-25T21:09:38.264 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-osd.2.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.25959.log 2024-08-25T21:09:38.264 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21627.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.21627.log.gz 2024-08-25T21:09:38.264 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.25612.log 2024-08-25T21:09:38.265 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.25959.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21839.log 2024-08-25T21:09:38.265 INFO:teuthology.orchestra.run.smithi119.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.25959.log.gz 2024-08-25T21:09:38.266 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.25612.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.25612.log.gz 2024-08-25T21:09:38.266 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.21881.log 2024-08-25T21:09:38.266 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21839.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21839.log.gz 2024-08-25T21:09:38.267 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/cluster1-client.admin.21881.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.21881.log.gz 2024-08-25T21:09:38.295 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55707.log 2024-08-25T21:09:38.295 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33135.log 2024-08-25T21:09:38.296 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55707.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63945.log 2024-08-25T21:09:38.296 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55707.log.gz 2024-08-25T21:09:38.297 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33135.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster2-client.admin.33135.log.gz --verbose 2024-08-25T21:09:38.297 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.76311.log 2024-08-25T21:09:38.298 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63945.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63945.log.gz 2024-08-25T21:09:38.298 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113063.log 2024-08-25T21:09:38.298 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76311.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.100355.log -- replaced with /var/log/ceph/cluster1-client.mirror.76311.log.gz 2024-08-25T21:09:38.298 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:38.299 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113063.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113063.log.gz 2024-08-25T21:09:38.299 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134590.log 2024-08-25T21:09:38.300 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100355.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100355.log.gz 2024-08-25T21:09:38.300 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72001.log 2024-08-25T21:09:38.301 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134590.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134590.log.gz 2024-08-25T21:09:38.301 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96766.log 2024-08-25T21:09:38.301 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72001.log.gz 2024-08-25T21:09:38.302 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49321.log 2024-08-25T21:09:38.302 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96766.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96766.log.gz 2024-08-25T21:09:38.302 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52916.log 2024-08-25T21:09:38.303 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49321.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49321.log.gz 2024-08-25T21:09:38.303 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76460.log 2024-08-25T21:09:38.304 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52916.log.gz 2024-08-25T21:09:38.304 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53924.log 2024-08-25T21:09:38.305 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76460.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76460.log.gz 2024-08-25T21:09:38.305 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113359.log 2024-08-25T21:09:38.305 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53924.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53924.log.gz 2024-08-25T21:09:38.306 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93485.log 2024-08-25T21:09:38.306 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113359.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113359.log.gz 2024-08-25T21:09:38.307 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93931.log 2024-08-25T21:09:38.307 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93485.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93485.log.gz 2024-08-25T21:09:38.307 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116399.log 2024-08-25T21:09:38.308 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.93931.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93931.log.gz 2024-08-25T21:09:38.308 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81629.log 2024-08-25T21:09:38.309 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116399.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116399.log.gz 2024-08-25T21:09:38.309 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126238.log 2024-08-25T21:09:38.309 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81629.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81629.log.gz 2024-08-25T21:09:38.310 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26380.log 2024-08-25T21:09:38.310 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.126238.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126238.log.gz 2024-08-25T21:09:38.310 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132022.log 2024-08-25T21:09:38.311 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26380.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26380.log.gz 2024-08-25T21:09:38.311 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108503.log 2024-08-25T21:09:38.312 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132022.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132022.log.gz 2024-08-25T21:09:38.312 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82585.log 2024-08-25T21:09:38.313 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108503.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108503.log.gz 2024-08-25T21:09:38.313 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29971.log 2024-08-25T21:09:38.313 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82585.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82585.log.gz 2024-08-25T21:09:38.314 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127846.log 2024-08-25T21:09:38.314 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29971.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29971.log.gz 2024-08-25T21:09:38.314 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33332.log 2024-08-25T21:09:38.315 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.127846.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127846.log.gz 2024-08-25T21:09:38.315 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131278.log 2024-08-25T21:09:38.316 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33332.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33332.log.gz 2024-08-25T21:09:38.316 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51573.log 2024-08-25T21:09:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131278.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131278.log.gz 2024-08-25T21:09:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75050.log 2024-08-25T21:09:38.317 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51573.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62326.log 2024-08-25T21:09:38.318 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75050.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75050.log.gz 2024-08-25T21:09:38.318 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43850.log 2024-08-25T21:09:38.319 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62326.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62326.log.gz 2024-08-25T21:09:38.319 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126358.log 2024-08-25T21:09:38.320 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43850.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43850.log.gz 2024-08-25T21:09:38.320 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133972.log 2024-08-25T21:09:38.321 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.126358.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126358.log.gz 2024-08-25T21:09:38.321 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76271.log 2024-08-25T21:09:38.322 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133972.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133972.log.gz 2024-08-25T21:09:38.322 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73136.log 2024-08-25T21:09:38.323 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76271.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76271.log.gz 2024-08-25T21:09:38.323 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68728.log 2024-08-25T21:09:38.323 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73136.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.73136.log.gz 2024-08-25T21:09:38.323 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.127494.log 2024-08-25T21:09:38.324 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.68728.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.68728.log.gz 2024-08-25T21:09:38.324 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38072.log 2024-08-25T21:09:38.325 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127494.log.gz 2024-08-25T21:09:38.325 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82746.log 2024-08-25T21:09:38.326 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38072.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38072.log.gz 2024-08-25T21:09:38.326 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37238.log 2024-08-25T21:09:38.327 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82746.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82746.log.gz 2024-08-25T21:09:38.327 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127320.log 2024-08-25T21:09:38.327 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37238.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114074.log 2024-08-25T21:09:38.328 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127320.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127320.log.gz 2024-08-25T21:09:38.328 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86107.log 2024-08-25T21:09:38.329 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114074.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114074.log.gz 2024-08-25T21:09:38.329 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57726.log 2024-08-25T21:09:38.330 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86107.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51573.log.gz 2024-08-25T21:09:38.330 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86107.log.gz 2024-08-25T21:09:38.330 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50783.log 2024-08-25T21:09:38.330 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57726.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57726.log.gz 2024-08-25T21:09:38.331 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123664.log 2024-08-25T21:09:38.331 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50783.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50783.log.gz 2024-08-25T21:09:38.331 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52203.log 2024-08-25T21:09:38.332 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123664.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123664.log.gz 2024-08-25T21:09:38.332 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52413.log 2024-08-25T21:09:38.333 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52203.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52203.log.gz 2024-08-25T21:09:38.333 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92862.log 2024-08-25T21:09:38.334 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52413.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52413.log.gz 2024-08-25T21:09:38.334 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106733.log 2024-08-25T21:09:38.334 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92862.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92862.log.gz 2024-08-25T21:09:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126944.log 2024-08-25T21:09:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106733.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106733.log.gz 2024-08-25T21:09:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86275.log 2024-08-25T21:09:38.336 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126944.log.gz 2024-08-25T21:09:38.336 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73576.log 2024-08-25T21:09:38.338 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86275.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37238.log.gz 2024-08-25T21:09:38.338 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86275.log.gz 2024-08-25T21:09:38.338 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39877.log 2024-08-25T21:09:38.338 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73576.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73576.log.gz 2024-08-25T21:09:38.338 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58700.log 2024-08-25T21:09:38.338 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39877.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39877.log.gz 2024-08-25T21:09:38.339 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123853.log 2024-08-25T21:09:38.339 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58700.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58700.log.gz 2024-08-25T21:09:38.339 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76584.log 2024-08-25T21:09:38.340 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123853.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123853.log.gz 2024-08-25T21:09:38.340 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50016.log 2024-08-25T21:09:38.341 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76584.log.gz 2024-08-25T21:09:38.341 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94481.log 2024-08-25T21:09:38.342 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50016.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50016.log.gz 2024-08-25T21:09:38.342 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111281.log 2024-08-25T21:09:38.342 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94481.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94481.log.gz 2024-08-25T21:09:38.343 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100229.log 2024-08-25T21:09:38.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111281.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111281.log.gz 2024-08-25T21:09:38.343 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141237.log 2024-08-25T21:09:38.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100229.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100229.log.gz 2024-08-25T21:09:38.344 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42231.log 2024-08-25T21:09:38.345 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141237.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141237.log.gz 2024-08-25T21:09:38.345 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49399.log 2024-08-25T21:09:38.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42231.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93218.log 2024-08-25T21:09:38.347 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42231.log.gz 2024-08-25T21:09:38.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49399.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49399.log.gz 2024-08-25T21:09:38.347 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.22703.log 2024-08-25T21:09:38.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93218.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93218.log.gz 2024-08-25T21:09:38.348 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48863.log 2024-08-25T21:09:38.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.22703.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.22703.log.gz 2024-08-25T21:09:38.348 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46997.log 2024-08-25T21:09:38.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48863.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48863.log.gz 2024-08-25T21:09:38.349 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66693.log 2024-08-25T21:09:38.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.46997.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46997.log.gz 2024-08-25T21:09:38.351 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122722.log 2024-08-25T21:09:38.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66693.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66693.log.gz 2024-08-25T21:09:38.351 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110472.log 2024-08-25T21:09:38.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122722.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122722.log.gz 2024-08-25T21:09:38.352 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56946.log 2024-08-25T21:09:38.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110472.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110472.log.gz 2024-08-25T21:09:38.352 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64786.log 2024-08-25T21:09:38.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.56946.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56946.log.gz 2024-08-25T21:09:38.353 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138146.log 2024-08-25T21:09:38.354 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64786.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64786.log.gz 2024-08-25T21:09:38.354 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48433.log 2024-08-25T21:09:38.355 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138146.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138146.log.gz 2024-08-25T21:09:38.355 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104223.log 2024-08-25T21:09:38.355 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.48433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48433.log.gz 2024-08-25T21:09:38.356 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58452.log 2024-08-25T21:09:38.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104223.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104223.log.gz 2024-08-25T21:09:38.356 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99534.log 2024-08-25T21:09:38.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58452.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58452.log.gz 2024-08-25T21:09:38.357 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110838.log 2024-08-25T21:09:38.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99534.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99534.log.gz 2024-08-25T21:09:38.358 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125977.log 2024-08-25T21:09:38.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110838.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110838.log.gz 2024-08-25T21:09:38.359 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29129.log 2024-08-25T21:09:38.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125977.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125977.log.gz 2024-08-25T21:09:38.361 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73916.log 2024-08-25T21:09:38.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29129.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29129.log.gz 2024-08-25T21:09:38.362 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63480.log 2024-08-25T21:09:38.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73916.log.gz 2024-08-25T21:09:38.362 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100988.log 2024-08-25T21:09:38.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63480.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63480.log.gz 2024-08-25T21:09:38.364 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47053.log 2024-08-25T21:09:38.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100988.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100988.log.gz 2024-08-25T21:09:38.364 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136964.log 2024-08-25T21:09:38.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47053.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47053.log.gz 2024-08-25T21:09:38.364 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119890.log 2024-08-25T21:09:38.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136964.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136964.log.gz 2024-08-25T21:09:38.365 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51363.log 2024-08-25T21:09:38.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119890.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119890.log.gz 2024-08-25T21:09:38.365 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26525.log 2024-08-25T21:09:38.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51363.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51363.log.gz 2024-08-25T21:09:38.366 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88775.log 2024-08-25T21:09:38.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26525.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26525.log.gz -5 2024-08-25T21:09:38.367 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.71126.log 2024-08-25T21:09:38.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88775.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88775.log.gz 2024-08-25T21:09:38.368 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.117966.log 2024-08-25T21:09:38.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71126.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71126.log.gz 2024-08-25T21:09:38.369 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90917.log 2024-08-25T21:09:38.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.117966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.117966.log.gz 2024-08-25T21:09:38.369 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104222.log 2024-08-25T21:09:38.370 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90917.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90917.log.gz 2024-08-25T21:09:38.370 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63356.log 2024-08-25T21:09:38.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104222.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104222.log.gz 2024-08-25T21:09:38.371 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97815.log 2024-08-25T21:09:38.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63356.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63356.log.gz 2024-08-25T21:09:38.372 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44437.log 2024-08-25T21:09:38.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97815.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97815.log.gz 2024-08-25T21:09:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88354.log 2024-08-25T21:09:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44437.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44437.log.gz 2024-08-25T21:09:38.373 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120348.log 2024-08-25T21:09:38.374 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88354.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88354.log.gz 2024-08-25T21:09:38.374 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58928.log 2024-08-25T21:09:38.375 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120348.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120348.log.gz 2024-08-25T21:09:38.375 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109822.log 2024-08-25T21:09:38.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58928.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58928.log.gz 2024-08-25T21:09:38.376 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35966.log 2024-08-25T21:09:38.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109822.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109822.log.gz 2024-08-25T21:09:38.377 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87095.log 2024-08-25T21:09:38.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35966.log.gz 2024-08-25T21:09:38.377 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64573.log 2024-08-25T21:09:38.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87095.log.gz 2024-08-25T21:09:38.378 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41979.log 2024-08-25T21:09:38.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64573.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110264.log 2024-08-25T21:09:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41979.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41979.log.gz 2024-08-25T21:09:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29401.log 2024-08-25T21:09:38.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110264.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110264.log.gz 2024-08-25T21:09:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65834.log 2024-08-25T21:09:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29401.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29401.log.gz 2024-08-25T21:09:38.381 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113000.log 2024-08-25T21:09:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65834.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65834.log.gz 2024-08-25T21:09:38.382 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135791.log 2024-08-25T21:09:38.383 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113000.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64573.log.gz 2024-08-25T21:09:38.383 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113000.log.gz 2024-08-25T21:09:38.383 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88606.log 2024-08-25T21:09:38.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135791.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39081.log 2024-08-25T21:09:38.384 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135791.log.gz 2024-08-25T21:09:38.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88606.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88606.log.gz 2024-08-25T21:09:38.385 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80928.log 2024-08-25T21:09:38.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39081.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39081.log.gz 2024-08-25T21:09:38.386 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.138049.log 2024-08-25T21:09:38.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80928.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80928.log.gz 2024-08-25T21:09:38.387 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98295.log 2024-08-25T21:09:38.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.138049.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.138049.log.gz 2024-08-25T21:09:38.387 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105577.log 2024-08-25T21:09:38.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98295.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98295.log.gz 2024-08-25T21:09:38.388 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82377.log 2024-08-25T21:09:38.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.105577.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105577.log.gz 2024-08-25T21:09:38.389 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115010.log 2024-08-25T21:09:38.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82377.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82377.log.gz 2024-08-25T21:09:38.390 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90640.log 2024-08-25T21:09:38.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115010.log.gz 2024-08-25T21:09:38.391 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123118.log 2024-08-25T21:09:38.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90640.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90640.log.gz 2024-08-25T21:09:38.391 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108625.log 2024-08-25T21:09:38.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123118.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123118.log.gz 2024-08-25T21:09:38.392 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142154.log 2024-08-25T21:09:38.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108625.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108625.log.gz 2024-08-25T21:09:38.393 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99955.log 2024-08-25T21:09:38.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.142154.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142154.log.gz 2024-08-25T21:09:38.394 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61777.log 2024-08-25T21:09:38.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99955.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99955.log.gz 2024-08-25T21:09:38.395 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107832.log 2024-08-25T21:09:38.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61777.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61777.log.gz 2024-08-25T21:09:38.395 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74776.log 2024-08-25T21:09:38.396 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107832.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107832.log.gz 2024-08-25T21:09:38.396 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37793.log 2024-08-25T21:09:38.397 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74776.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74776.log.gz 2024-08-25T21:09:38.397 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72219.log 2024-08-25T21:09:38.398 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37793.log.gz 2024-08-25T21:09:38.398 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100147.log 2024-08-25T21:09:38.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72219.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72219.log.gz 2024-08-25T21:09:38.399 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88711.log 2024-08-25T21:09:38.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100147.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100147.log.gz 2024-08-25T21:09:38.399 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.33080.log 2024-08-25T21:09:38.400 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88711.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88711.log.gz 2024-08-25T21:09:38.400 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.128751.log 2024-08-25T21:09:38.401 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.33080.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.33080.log.gz 2024-08-25T21:09:38.401 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102013.log 2024-08-25T21:09:38.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.128751.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.128751.log.gz 2024-08-25T21:09:38.402 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26295.log 2024-08-25T21:09:38.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102013.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102013.log.gz 2024-08-25T21:09:38.403 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93564.log 2024-08-25T21:09:38.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26295.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26295.log.gz 2024-08-25T21:09:38.404 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135010.log 2024-08-25T21:09:38.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93564.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93564.log.gz 2024-08-25T21:09:38.404 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73410.log 2024-08-25T21:09:38.405 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135010.log.gz 2024-08-25T21:09:38.405 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56542.log 2024-08-25T21:09:38.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73410.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73410.log.gz 2024-08-25T21:09:38.406 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52746.log 2024-08-25T21:09:38.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56542.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56542.log.gz 2024-08-25T21:09:38.407 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111553.log 2024-08-25T21:09:38.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52746.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52746.log.gz 2024-08-25T21:09:38.408 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106562.log 2024-08-25T21:09:38.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111553.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111553.log.gz 2024-08-25T21:09:38.408 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126515.log 2024-08-25T21:09:38.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.106562.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106562.log.gz 2024-08-25T21:09:38.409 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29885.log 2024-08-25T21:09:38.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126515.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126515.log.gz 2024-08-25T21:09:38.410 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65205.log 2024-08-25T21:09:38.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29885.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29885.log.gz 2024-08-25T21:09:38.411 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67540.log 2024-08-25T21:09:38.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65205.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65205.log.gz 2024-08-25T21:09:38.412 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115430.log 2024-08-25T21:09:38.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67540.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67540.log.gz 2024-08-25T21:09:38.412 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75262.log 2024-08-25T21:09:38.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115430.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.admin.115430.log.gz 2024-08-25T21:09:38.413 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster2-client.admin.45218.log 2024-08-25T21:09:38.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75262.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75262.log.gz 2024-08-25T21:09:38.414 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97921.log 2024-08-25T21:09:38.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45218.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45218.log.gz 2024-08-25T21:09:38.415 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105019.log 2024-08-25T21:09:38.416 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97921.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97921.log.gz 2024-08-25T21:09:38.416 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46976.log 2024-08-25T21:09:38.416 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105019.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105019.log.gz 2024-08-25T21:09:38.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69962.log 2024-08-25T21:09:38.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46976.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46976.log.gz 2024-08-25T21:09:38.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115695.log 2024-08-25T21:09:38.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.69962.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69962.log.gz 2024-08-25T21:09:38.418 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74462.log 2024-08-25T21:09:38.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115695.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115695.log.gz 2024-08-25T21:09:38.419 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41241.log 2024-08-25T21:09:38.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74462.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74462.log.gz 2024-08-25T21:09:38.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114534.log 2024-08-25T21:09:38.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41241.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41241.log.gz 2024-08-25T21:09:38.421 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123984.log 2024-08-25T21:09:38.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114534.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114534.log.gz 2024-08-25T21:09:38.421 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83861.log 2024-08-25T21:09:38.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123984.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123984.log.gz 2024-08-25T21:09:38.422 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92222.log 2024-08-25T21:09:38.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83861.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83861.log.gz 2024-08-25T21:09:38.423 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52160.log 2024-08-25T21:09:38.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.92222.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113147.log 2024-08-25T21:09:38.424 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.92222.log.gz 2024-08-25T21:09:38.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52160.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52160.log.gz 2024-08-25T21:09:38.425 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33656.log 2024-08-25T21:09:38.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113147.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113147.log.gz 2024-08-25T21:09:38.426 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138878.log 2024-08-25T21:09:38.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33656.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33656.log.gz 2024-08-25T21:09:38.427 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84188.log 2024-08-25T21:09:38.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138878.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138878.log.gz 2024-08-25T21:09:38.427 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62850.log 2024-08-25T21:09:38.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84188.log.gz 2024-08-25T21:09:38.428 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52578.log 2024-08-25T21:09:38.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62850.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62850.log.gz 2024-08-25T21:09:38.429 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59341.log 2024-08-25T21:09:38.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52578.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52578.log.gz 2024-08-25T21:09:38.430 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60013.log 2024-08-25T21:09:38.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59341.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59341.log.gz 2024-08-25T21:09:38.430 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134116.log 2024-08-25T21:09:38.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60013.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60013.log.gz 2024-08-25T21:09:38.431 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53506.log 2024-08-25T21:09:38.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134116.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134116.log.gz 2024-08-25T21:09:38.432 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85205.log 2024-08-25T21:09:38.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53506.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53506.log.gz 2024-08-25T21:09:38.433 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86397.log 2024-08-25T21:09:38.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85205.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85205.log.gz 2024-08-25T21:09:38.434 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25736.log 2024-08-25T21:09:38.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86397.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86397.log.gz 2024-08-25T21:09:38.434 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85766.log 2024-08-25T21:09:38.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25736.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25736.log.gz 2024-08-25T21:09:38.438 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93616.log 2024-08-25T21:09:38.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85766.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85766.log.gz 2024-08-25T21:09:38.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37547.log 2024-08-25T21:09:38.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.93616.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93616.log.gz 2024-08-25T21:09:38.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57817.log 2024-08-25T21:09:38.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37547.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37547.log.gz 2024-08-25T21:09:38.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87305.log 2024-08-25T21:09:38.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57817.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57817.log.gz 2024-08-25T21:09:38.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51780.log 2024-08-25T21:09:38.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87305.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87305.log.gz 2024-08-25T21:09:38.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31677.log 2024-08-25T21:09:38.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51780.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51780.log.gz 2024-08-25T21:09:38.442 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36550.log 2024-08-25T21:09:38.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31677.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31677.log.gz 2024-08-25T21:09:38.442 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89339.log 2024-08-25T21:09:38.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36550.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36550.log.gz 2024-08-25T21:09:38.443 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49722.log 2024-08-25T21:09:38.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89339.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89339.log.gz 2024-08-25T21:09:38.443 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122068.log 2024-08-25T21:09:38.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49722.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49722.log.gz 2024-08-25T21:09:38.444 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40339.log 2024-08-25T21:09:38.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122068.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122068.log.gz 2024-08-25T21:09:38.444 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43741.log 2024-08-25T21:09:38.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40339.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40339.log.gz 2024-08-25T21:09:38.445 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51110.log 2024-08-25T21:09:38.446 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43741.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43741.log.gz 2024-08-25T21:09:38.446 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108626.log 2024-08-25T21:09:38.446 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51110.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51110.log.gz 2024-08-25T21:09:38.447 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76859.log 2024-08-25T21:09:38.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108626.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108626.log.gz 2024-08-25T21:09:38.447 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42188.log 2024-08-25T21:09:38.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76859.log.gz 2024-08-25T21:09:38.448 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76391.log 2024-08-25T21:09:38.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42188.log.gz 2024-08-25T21:09:38.449 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39373.log 2024-08-25T21:09:38.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76391.log.gz 2024-08-25T21:09:38.450 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130529.log 2024-08-25T21:09:38.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39373.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39373.log.gz 2024-08-25T21:09:38.451 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23813.log 2024-08-25T21:09:38.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130529.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130529.log.gz 2024-08-25T21:09:38.451 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80250.log 2024-08-25T21:09:38.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23813.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23813.log.gz 2024-08-25T21:09:38.452 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116701.log 2024-08-25T21:09:38.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80250.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80250.log.gz 2024-08-25T21:09:38.453 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51234.log 2024-08-25T21:09:38.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.116701.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.116701.log.gz 2024-08-25T21:09:38.454 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137322.log 2024-08-25T21:09:38.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51234.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51234.log.gz 2024-08-25T21:09:38.455 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124504.log 2024-08-25T21:09:38.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137322.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137322.log.gz 2024-08-25T21:09:38.456 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104620.log 2024-08-25T21:09:38.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124504.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124504.log.gz 2024-08-25T21:09:38.456 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90691.log 2024-08-25T21:09:38.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.104620.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104620.log.gz 2024-08-25T21:09:38.457 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-osd.1.log 2024-08-25T21:09:38.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90691.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90691.log.gz 2024-08-25T21:09:38.458 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49137.log 2024-08-25T21:09:38.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-osd.1.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120347.log 2024-08-25T21:09:38.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49137.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49137.log.gz 2024-08-25T21:09:38.460 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48086.log 2024-08-25T21:09:38.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120347.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120347.log.gz 2024-08-25T21:09:38.460 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91424.log 2024-08-25T21:09:38.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.48086.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48086.log.gz 2024-08-25T21:09:38.461 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118059.log 2024-08-25T21:09:38.462 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91424.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91424.log.gz 2024-08-25T21:09:38.462 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77769.log 2024-08-25T21:09:38.463 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.118059.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118059.log.gz 2024-08-25T21:09:38.463 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40384.log 2024-08-25T21:09:38.464 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77769.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77769.log.gz 2024-08-25T21:09:38.464 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126467.log 2024-08-25T21:09:38.464 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40384.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40384.log.gz 2024-08-25T21:09:38.464 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122594.log 2024-08-25T21:09:38.465 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.126467.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126467.log.gz 2024-08-25T21:09:38.465 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22352.log 2024-08-25T21:09:38.466 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122594.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122594.log.gz 2024-08-25T21:09:38.466 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90398.log 2024-08-25T21:09:38.467 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22352.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92275.log 2024-08-25T21:09:38.467 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22352.log.gz 2024-08-25T21:09:38.468 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90398.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90398.log.gz 2024-08-25T21:09:38.468 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132353.log 2024-08-25T21:09:38.468 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92275.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92275.log.gz 2024-08-25T21:09:38.469 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97118.log 2024-08-25T21:09:38.469 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132353.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132353.log.gz 2024-08-25T21:09:38.469 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97143.log 2024-08-25T21:09:38.470 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97118.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97118.log.gz 2024-08-25T21:09:38.470 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121398.log 2024-08-25T21:09:38.471 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97143.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97143.log.gz 2024-08-25T21:09:38.471 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90175.log 2024-08-25T21:09:38.472 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121398.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121398.log.gz 2024-08-25T21:09:38.472 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117136.log 2024-08-25T21:09:38.472 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90175.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90175.log.gz 2024-08-25T21:09:38.473 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117190.log 2024-08-25T21:09:38.473 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117136.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117136.log.gz 2024-08-25T21:09:38.473 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125915.log 2024-08-25T21:09:38.474 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117190.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117190.log.gz 2024-08-25T21:09:38.474 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58566.log 2024-08-25T21:09:38.475 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125915.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125915.log.gz 2024-08-25T21:09:38.475 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47391.log 2024-08-25T21:09:38.476 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58566.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58566.log.gz 2024-08-25T21:09:38.476 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.117738.log 2024-08-25T21:09:38.476 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47391.log.gz 2024-08-25T21:09:38.477 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135153.log 2024-08-25T21:09:38.477 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.117738.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.117738.log.gz 2024-08-25T21:09:38.477 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69659.log 2024-08-25T21:09:38.478 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135153.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135153.log.gz 2024-08-25T21:09:38.478 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82108.log 2024-08-25T21:09:38.479 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69659.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69659.log.gz 2024-08-25T21:09:38.479 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61088.log 2024-08-25T21:09:38.480 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82108.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82108.log.gz 2024-08-25T21:09:38.480 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106106.log 2024-08-25T21:09:38.480 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61088.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61088.log.gz 2024-08-25T21:09:38.481 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99054.log 2024-08-25T21:09:38.481 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.106106.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106106.log.gz 2024-08-25T21:09:38.481 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139696.log 2024-08-25T21:09:38.482 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99054.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99054.log.gz 2024-08-25T21:09:38.482 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93088.log 2024-08-25T21:09:38.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.139696.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139696.log.gz 2024-08-25T21:09:38.483 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71568.log 2024-08-25T21:09:38.484 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.93088.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93088.log.gz 2024-08-25T21:09:38.484 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120176.log 2024-08-25T21:09:38.485 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71568.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71568.log.gz 2024-08-25T21:09:38.485 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79284.log 2024-08-25T21:09:38.485 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120176.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120176.log.gz 2024-08-25T21:09:38.486 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47211.log 2024-08-25T21:09:38.486 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79284.log.gz 2024-08-25T21:09:38.486 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129113.log 2024-08-25T21:09:38.487 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.47211.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47211.log.gz 2024-08-25T21:09:38.487 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28771.log 2024-08-25T21:09:38.488 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.129113.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129113.log.gz 2024-08-25T21:09:38.488 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.91852.log 2024-08-25T21:09:38.489 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28771.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28771.log.gz 2024-08-25T21:09:38.489 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.32364.log 2024-08-25T21:09:38.489 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.91852.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.91852.log.gz 2024-08-25T21:09:38.489 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139581.log 2024-08-25T21:09:38.490 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.32364.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.32364.log.gz 2024-08-25T21:09:38.490 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27681.log 2024-08-25T21:09:38.491 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139581.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139581.log.gz 2024-08-25T21:09:38.491 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86548.log 2024-08-25T21:09:38.492 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27681.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27681.log.gz 2024-08-25T21:09:38.492 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135640.log 2024-08-25T21:09:38.493 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86548.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86548.log.gz 2024-08-25T21:09:38.493 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23979.log 2024-08-25T21:09:38.493 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135640.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135640.log.gz 2024-08-25T21:09:38.493 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65708.log 2024-08-25T21:09:38.494 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23979.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23979.log.gz 2024-08-25T21:09:38.494 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47475.log 2024-08-25T21:09:38.495 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65708.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65708.log.gz 2024-08-25T21:09:38.495 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48592.log 2024-08-25T21:09:38.496 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.47475.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47475.log.gz 2024-08-25T21:09:38.496 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29045.log 2024-08-25T21:09:38.496 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48592.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48592.log.gz 2024-08-25T21:09:38.497 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102549.log 2024-08-25T21:09:38.497 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29045.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29045.log.gz 2024-08-25T21:09:38.497 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47559.log 2024-08-25T21:09:38.498 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102549.log.gz 2024-08-25T21:09:38.498 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81372.log 2024-08-25T21:09:38.499 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47559.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47559.log.gz 2024-08-25T21:09:38.499 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127478.log 2024-08-25T21:09:38.500 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81372.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81372.log.gz 2024-08-25T21:09:38.500 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41434.log 2024-08-25T21:09:38.501 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127478.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127478.log.gz 2024-08-25T21:09:38.501 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57180.log 2024-08-25T21:09:38.502 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41434.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41434.log.gz 2024-08-25T21:09:38.502 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75768.log 2024-08-25T21:09:38.502 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57180.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57180.log.gz 2024-08-25T21:09:38.502 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62730.log 2024-08-25T21:09:38.503 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75768.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.75768.log.gz -- 2024-08-25T21:09:38.503 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.123164.log 2024-08-25T21:09:38.504 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62730.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62730.log.gz 2024-08-25T21:09:38.504 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78361.log 2024-08-25T21:09:38.504 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123164.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123164.log.gz 2024-08-25T21:09:38.505 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110272.log 2024-08-25T21:09:38.505 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78361.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78361.log.gz 2024-08-25T21:09:38.505 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107446.log 2024-08-25T21:09:38.506 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110272.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110272.log.gz 2024-08-25T21:09:38.506 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26273.log 2024-08-25T21:09:38.507 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107446.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107446.log.gz 2024-08-25T21:09:38.507 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74572.log 2024-08-25T21:09:38.508 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26273.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26273.log.gz 2024-08-25T21:09:38.508 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69434.log 2024-08-25T21:09:38.508 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74572.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132960.log 2024-08-25T21:09:38.509 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74572.log.gz 2024-08-25T21:09:38.509 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.69434.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69434.log.gz 2024-08-25T21:09:38.509 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98296.log 2024-08-25T21:09:38.510 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132960.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132960.log.gz 2024-08-25T21:09:38.510 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60753.log 2024-08-25T21:09:38.511 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98296.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98296.log.gz 2024-08-25T21:09:38.511 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53772.log 2024-08-25T21:09:38.512 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60753.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60753.log.gz 2024-08-25T21:09:38.512 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65246.log 2024-08-25T21:09:38.512 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53772.log.gz 2024-08-25T21:09:38.512 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70558.log 2024-08-25T21:09:38.513 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65246.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65246.log.gz 2024-08-25T21:09:38.513 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30815.log 2024-08-25T21:09:38.514 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70558.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70558.log.gz 2024-08-25T21:09:38.514 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85035.log 2024-08-25T21:09:38.515 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.30815.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30815.log.gz 2024-08-25T21:09:38.515 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83443.log 2024-08-25T21:09:38.516 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85035.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85035.log.gz 2024-08-25T21:09:38.516 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120810.log 2024-08-25T21:09:38.516 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.83443.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83443.log.gz 2024-08-25T21:09:38.517 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88249.log 2024-08-25T21:09:38.517 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120810.log.gz 2024-08-25T21:09:38.518 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67757.log 2024-08-25T21:09:38.518 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88249.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88249.log.gz 2024-08-25T21:09:38.518 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86485.log 2024-08-25T21:09:38.519 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67757.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67757.log.gz 2024-08-25T21:09:38.519 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132442.log 2024-08-25T21:09:38.520 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86485.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86485.log.gz 2024-08-25T21:09:38.520 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26379.log 2024-08-25T21:09:38.520 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132442.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132442.log.gz 2024-08-25T21:09:38.520 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132290.log 2024-08-25T21:09:38.521 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26379.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26379.log.gz 2024-08-25T21:09:38.521 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122196.log 2024-08-25T21:09:38.522 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132290.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132290.log.gz 2024-08-25T21:09:38.522 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37568.log 2024-08-25T21:09:38.523 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122196.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122196.log.gz 2024-08-25T21:09:38.523 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.35081.log 2024-08-25T21:09:38.524 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37568.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.37568.log.gz 2024-08-25T21:09:38.524 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.91578.log 2024-08-25T21:09:38.525 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.35081.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.35081.log.gz 2024-08-25T21:09:38.525 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104597.log 2024-08-25T21:09:38.525 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91578.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134592.log 2024-08-25T21:09:38.525 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91578.log.gz 2024-08-25T21:09:38.527 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.104597.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104597.log.gz 2024-08-25T21:09:38.527 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52665.log 2024-08-25T21:09:38.527 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134592.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134592.log.gz 2024-08-25T21:09:38.527 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136294.log 2024-08-25T21:09:38.528 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52665.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52665.log.gz 2024-08-25T21:09:38.528 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138724.log 2024-08-25T21:09:38.529 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136294.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24394.log 2024-08-25T21:09:38.529 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136294.log.gz 2024-08-25T21:09:38.529 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138724.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138724.log.gz 2024-08-25T21:09:38.530 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71963.log 2024-08-25T21:09:38.530 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24394.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.53882.log 2024-08-25T21:09:38.530 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.24394.log.gz 2024-08-25T21:09:38.531 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71963.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71963.log.gz 2024-08-25T21:09:38.531 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82979.log 2024-08-25T21:09:38.532 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53882.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53882.log.gz 2024-08-25T21:09:38.532 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140804.log 2024-08-25T21:09:38.533 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82979.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29404.log 2024-08-25T21:09:38.533 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82979.log.gz 2024-08-25T21:09:38.533 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140804.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140804.log.gz 2024-08-25T21:09:38.533 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24285.log 2024-08-25T21:09:38.534 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29404.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.29404.log.gz 2024-08-25T21:09:38.534 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101932.log 2024-08-25T21:09:38.535 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24285.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.24285.log.gz 2024-08-25T21:09:38.535 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.90511.log 2024-08-25T21:09:38.535 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101932.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101932.log.gz 2024-08-25T21:09:38.536 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127252.log 2024-08-25T21:09:38.536 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90511.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90511.log.gz 2024-08-25T21:09:38.536 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77531.log 2024-08-25T21:09:38.537 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127252.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127252.log.gz 2024-08-25T21:09:38.537 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132440.log 2024-08-25T21:09:38.538 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77531.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77531.log.gz 2024-08-25T21:09:38.538 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130268.log 2024-08-25T21:09:38.539 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132440.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132440.log.gz 2024-08-25T21:09:38.539 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105858.log 2024-08-25T21:09:38.539 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.130268.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.130268.log.gz -5 2024-08-25T21:09:38.540 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.81860.log 2024-08-25T21:09:38.540 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105858.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105858.log.gz 2024-08-25T21:09:38.540 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28203.log 2024-08-25T21:09:38.541 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.81860.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81860.log.gz 2024-08-25T21:09:38.541 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79460.log 2024-08-25T21:09:38.542 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28203.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28203.log.gz 2024-08-25T21:09:38.542 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84470.log 2024-08-25T21:09:38.543 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79460.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79460.log.gz 2024-08-25T21:09:38.543 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81616.log 2024-08-25T21:09:38.544 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84470.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84470.log.gz 2024-08-25T21:09:38.544 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126345.log 2024-08-25T21:09:38.544 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81616.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81616.log.gz 2024-08-25T21:09:38.545 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65177.log 2024-08-25T21:09:38.545 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126345.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126345.log.gz 2024-08-25T21:09:38.546 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106630.log 2024-08-25T21:09:38.546 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65177.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109339.log 2024-08-25T21:09:38.546 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65177.log.gz 2024-08-25T21:09:38.547 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106630.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106630.log.gz 2024-08-25T21:09:38.547 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72051.log 2024-08-25T21:09:38.548 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109339.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109339.log.gz 2024-08-25T21:09:38.548 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89259.log 2024-08-25T21:09:38.548 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72051.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72051.log.gz 2024-08-25T21:09:38.549 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124814.log 2024-08-25T21:09:38.549 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89259.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89259.log.gz 2024-08-25T21:09:38.549 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83491.log 2024-08-25T21:09:38.550 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124814.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.124814.log.gz 2024-08-25T21:09:38.550 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.67626.log 2024-08-25T21:09:38.551 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83491.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.83491.log.gz 2024-08-25T21:09:38.551 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74571.log 2024-08-25T21:09:38.551 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67626.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67626.log.gz 2024-08-25T21:09:38.551 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48926.log 2024-08-25T21:09:38.552 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74571.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74571.log.gz 2024-08-25T21:09:38.552 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65372.log 2024-08-25T21:09:38.553 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48926.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54672.log 2024-08-25T21:09:38.553 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48926.log.gz 2024-08-25T21:09:38.554 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65372.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.mirror.65372.log.gz /var/log/ceph/cluster1-client.admin.54531.log 2024-08-25T21:09:38.554 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:38.554 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54672.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75411.log 2024-08-25T21:09:38.555 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54672.log.gz 2024-08-25T21:09:38.555 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54531.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54531.log.gz 2024-08-25T21:09:38.555 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.129336.log 2024-08-25T21:09:38.557 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75411.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75411.log.gz 2024-08-25T21:09:38.557 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45145.log 2024-08-25T21:09:38.557 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.129336.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.129336.log.gz 2024-08-25T21:09:38.557 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49825.log 2024-08-25T21:09:38.558 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45145.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45145.log.gz 2024-08-25T21:09:38.558 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34323.log 2024-08-25T21:09:38.558 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49825.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49825.log.gz 2024-08-25T21:09:38.558 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36272.log 2024-08-25T21:09:38.559 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.34323.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34323.log.gz 2024-08-25T21:09:38.559 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72028.log 2024-08-25T21:09:38.560 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36272.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36272.log.gz 2024-08-25T21:09:38.560 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39037.log 2024-08-25T21:09:38.560 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72028.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72028.log.gz 2024-08-25T21:09:38.560 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111427.log 2024-08-25T21:09:38.561 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39037.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39037.log.gz 2024-08-25T21:09:38.561 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23916.log 2024-08-25T21:09:38.562 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111427.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111427.log.gz 2024-08-25T21:09:38.562 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60257.log 2024-08-25T21:09:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23916.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87785.log 2024-08-25T21:09:38.563 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23916.log.gz 2024-08-25T21:09:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60257.log.gz 2024-08-25T21:09:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86317.log 2024-08-25T21:09:38.564 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87785.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87785.log.gz 2024-08-25T21:09:38.564 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88857.log 2024-08-25T21:09:38.565 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86317.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86317.log.gz 2024-08-25T21:09:38.565 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117492.log 2024-08-25T21:09:38.566 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88857.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88857.log.gz 2024-08-25T21:09:38.566 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132503.log 2024-08-25T21:09:38.567 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117492.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49514.log 2024-08-25T21:09:38.567 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117492.log.gz 2024-08-25T21:09:38.567 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132503.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132503.log.gz 2024-08-25T21:09:38.567 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100252.log 2024-08-25T21:09:38.568 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49514.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49514.log.gz 2024-08-25T21:09:38.568 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.142885.log 2024-08-25T21:09:38.569 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100252.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100252.log.gz 2024-08-25T21:09:38.569 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105093.log 2024-08-25T21:09:38.570 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.142885.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.142885.log.gz 2024-08-25T21:09:38.570 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46044.log 2024-08-25T21:09:38.570 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105093.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105093.log.gz 2024-08-25T21:09:38.570 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139723.log 2024-08-25T21:09:38.571 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46044.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46044.log.gz 2024-08-25T21:09:38.571 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30393.log 2024-08-25T21:09:38.572 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.139723.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139723.log.gz 2024-08-25T21:09:38.572 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.96167.log 2024-08-25T21:09:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30393.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30393.log.gz 2024-08-25T21:09:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105458.log 2024-08-25T21:09:38.574 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.96167.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.96167.log.gz 2024-08-25T21:09:38.574 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24721.log 2024-08-25T21:09:38.574 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105458.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.23445.log 2024-08-25T21:09:38.575 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.105458.log.gz 2024-08-25T21:09:38.575 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24721.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24721.log.gz 2024-08-25T21:09:38.575 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83700.log 2024-08-25T21:09:38.576 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.23445.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.23445.log.gz 2024-08-25T21:09:38.576 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136626.log 2024-08-25T21:09:38.577 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83700.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83700.log.gz 2024-08-25T21:09:38.577 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.117852.log 2024-08-25T21:09:38.578 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136626.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136626.log.gz 2024-08-25T21:09:38.578 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53586.log 2024-08-25T21:09:38.578 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.117852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.117852.log.gz 2024-08-25T21:09:38.578 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132940.log 2024-08-25T21:09:38.579 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53586.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53586.log.gz 2024-08-25T21:09:38.579 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80896.log 2024-08-25T21:09:38.580 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132940.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132940.log.gz 2024-08-25T21:09:38.580 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71174.log 2024-08-25T21:09:38.580 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80896.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70122.log 2024-08-25T21:09:38.581 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80896.log.gz 2024-08-25T21:09:38.581 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.71174.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38533.log 2024-08-25T21:09:38.581 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71174.log.gz 2024-08-25T21:09:38.582 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70122.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70122.log.gz 2024-08-25T21:09:38.582 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142040.log 2024-08-25T21:09:38.583 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38533.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38533.log.gz 2024-08-25T21:09:38.583 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77824.log 2024-08-25T21:09:38.584 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.142040.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142040.log.gz 2024-08-25T21:09:38.584 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33950.log 2024-08-25T21:09:38.585 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77824.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77824.log.gz 2024-08-25T21:09:38.585 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81948.log 2024-08-25T21:09:38.585 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33950.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33950.log.gz 2024-08-25T21:09:38.585 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78099.log 2024-08-25T21:09:38.586 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.81948.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81948.log.gz 2024-08-25T21:09:38.586 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107373.log 2024-08-25T21:09:38.587 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78099.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106288.log 2024-08-25T21:09:38.587 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78099.log.gz 2024-08-25T21:09:38.588 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107373.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107373.log.gz 2024-08-25T21:09:38.588 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72487.log 2024-08-25T21:09:38.588 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106288.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106288.log.gz 2024-08-25T21:09:38.588 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48925.log 2024-08-25T21:09:38.589 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72487.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72487.log.gzgzip 2024-08-25T21:09:38.589 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50688.log 2024-08-25T21:09:38.590 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48925.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48925.log.gz 2024-08-25T21:09:38.590 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87682.log 2024-08-25T21:09:38.591 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50688.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50688.log.gz 2024-08-25T21:09:38.591 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98087.log 2024-08-25T21:09:38.591 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87682.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87682.log.gz 2024-08-25T21:09:38.591 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113967.log 2024-08-25T21:09:38.592 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98087.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98087.log.gz 2024-08-25T21:09:38.592 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84888.log 2024-08-25T21:09:38.593 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113967.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113967.log.gz 2024-08-25T21:09:38.593 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59995.log 2024-08-25T21:09:38.594 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84888.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84888.log.gz 2024-08-25T21:09:38.594 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90010.log 2024-08-25T21:09:38.594 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59995.log.gz 2024-08-25T21:09:38.594 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100462.log 2024-08-25T21:09:38.595 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90010.log.gz 2024-08-25T21:09:38.595 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25105.log 2024-08-25T21:09:38.596 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100462.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100462.log.gz 2024-08-25T21:09:38.596 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135012.log 2024-08-25T21:09:38.597 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25105.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89380.log 2024-08-25T21:09:38.597 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25105.log.gz 2024-08-25T21:09:38.597 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135012.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135012.log.gz 2024-08-25T21:09:38.597 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77002.log 2024-08-25T21:09:38.598 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89380.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89380.log.gzgzip 2024-08-25T21:09:38.598 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94969.log 2024-08-25T21:09:38.599 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77002.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77002.log.gz 2024-08-25T21:09:38.599 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120284.log 2024-08-25T21:09:38.600 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94969.log.gz 2024-08-25T21:09:38.600 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103998.log 2024-08-25T21:09:38.600 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120284.log.gz 2024-08-25T21:09:38.600 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124424.log 2024-08-25T21:09:38.601 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.103998.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92863.log 2024-08-25T21:09:38.601 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.103998.log.gz 2024-08-25T21:09:38.602 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124424.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124424.log.gz 2024-08-25T21:09:38.602 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99495.log 2024-08-25T21:09:38.602 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92863.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102261.log 2024-08-25T21:09:38.603 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92863.log.gz 2024-08-25T21:09:38.603 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99495.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99495.log.gz 2024-08-25T21:09:38.603 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66800.log 2024-08-25T21:09:38.604 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102261.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102261.log.gz 2024-08-25T21:09:38.604 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66907.log 2024-08-25T21:09:38.605 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66800.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67537.log 2024-08-25T21:09:38.605 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66800.log.gz 2024-08-25T21:09:38.605 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66907.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66907.log.gz 2024-08-25T21:09:38.606 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42105.log 2024-08-25T21:09:38.606 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67537.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67537.log.gz 2024-08-25T21:09:38.606 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32029.log 2024-08-25T21:09:38.607 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42105.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42105.log.gz 2024-08-25T21:09:38.607 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47126.log 2024-08-25T21:09:38.608 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32029.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32029.log.gz 2024-08-25T21:09:38.608 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101961.log 2024-08-25T21:09:38.608 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47126.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47126.log.gz 2024-08-25T21:09:38.608 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53334.log 2024-08-25T21:09:38.609 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101961.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101961.log.gz 2024-08-25T21:09:38.609 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.32804.log 2024-08-25T21:09:38.610 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53334.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53334.log.gz 2024-08-25T21:09:38.610 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64908.log 2024-08-25T21:09:38.611 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.32804.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32804.log.gz 2024-08-25T21:09:38.611 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100528.log 2024-08-25T21:09:38.612 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64908.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64908.log.gz 2024-08-25T21:09:38.612 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115210.log 2024-08-25T21:09:38.612 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100528.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100528.log.gz 2024-08-25T21:09:38.612 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107374.log 2024-08-25T21:09:38.613 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115210.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115210.log.gz 2024-08-25T21:09:38.613 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64154.log 2024-08-25T21:09:38.614 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107374.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.107374.log.gz /var/log/ceph/cluster1-client.mirror.51866.log 2024-08-25T21:09:38.614 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:38.615 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64154.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64154.log.gz 2024-08-25T21:09:38.615 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89680.log 2024-08-25T21:09:38.616 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51866.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51866.log.gz 2024-08-25T21:09:38.616 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73915.log 2024-08-25T21:09:38.616 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89680.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89680.log.gz 2024-08-25T21:09:38.616 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45230.log 2024-08-25T21:09:38.617 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73915.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73915.log.gz 2024-08-25T21:09:38.617 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133930.log 2024-08-25T21:09:38.618 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45230.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45230.log.gz 2024-08-25T21:09:38.618 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72911.log 2024-08-25T21:09:38.619 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133930.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133930.log.gz 2024-08-25T21:09:38.619 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118080.log 2024-08-25T21:09:38.620 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72911.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72911.log.gz 2024-08-25T21:09:38.620 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41137.log 2024-08-25T21:09:38.620 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118080.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118080.log.gz 2024-08-25T21:09:38.620 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108712.log 2024-08-25T21:09:38.621 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41137.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66600.log 2024-08-25T21:09:38.621 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41137.log.gz 2024-08-25T21:09:38.622 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108712.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108712.log.gz 2024-08-25T21:09:38.622 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112335.log 2024-08-25T21:09:38.623 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66600.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66600.log.gz 2024-08-25T21:09:38.623 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23135.log 2024-08-25T21:09:38.624 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112335.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112335.log.gz 2024-08-25T21:09:38.624 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121208.log 2024-08-25T21:09:38.624 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23135.log.gz 2024-08-25T21:09:38.625 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68438.log 2024-08-25T21:09:38.625 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121208.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121208.log.gz 2024-08-25T21:09:38.625 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47016.log 2024-08-25T21:09:38.626 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68438.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68438.log.gz 2024-08-25T21:09:38.626 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120999.log 2024-08-25T21:09:38.627 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47016.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47016.log.gz 2024-08-25T21:09:38.627 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107043.log 2024-08-25T21:09:38.628 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120999.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120999.log.gz 2024-08-25T21:09:38.628 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45865.log 2024-08-25T21:09:38.628 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.107043.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136962.log 2024-08-25T21:09:38.629 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107043.log.gz 2024-08-25T21:09:38.629 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45865.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45865.log.gz 2024-08-25T21:09:38.629 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115694.log 2024-08-25T21:09:38.630 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136962.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136962.log.gz 2024-08-25T21:09:38.630 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115518.log 2024-08-25T21:09:38.631 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115694.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115694.log.gz 2024-08-25T21:09:38.631 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37693.log 2024-08-25T21:09:38.632 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115518.log.gz 2024-08-25T21:09:38.632 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125324.log 2024-08-25T21:09:38.632 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37693.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37693.log.gz -5 2024-08-25T21:09:38.632 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.50458.log 2024-08-25T21:09:38.633 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125324.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125324.log.gz 2024-08-25T21:09:38.633 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37001.log 2024-08-25T21:09:38.634 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50458.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50458.log.gz 2024-08-25T21:09:38.634 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53166.log 2024-08-25T21:09:38.634 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37001.log.gz 2024-08-25T21:09:38.635 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98762.log 2024-08-25T21:09:38.635 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53166.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53166.log.gz 2024-08-25T21:09:38.635 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109254.log 2024-08-25T21:09:38.636 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98762.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98762.log.gz 2024-08-25T21:09:38.636 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122552.log 2024-08-25T21:09:38.637 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109254.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109254.log.gz 2024-08-25T21:09:38.637 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45223.log 2024-08-25T21:09:38.638 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122552.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122552.log.gz 2024-08-25T21:09:38.638 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113824.log 2024-08-25T21:09:38.638 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45223.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45223.log.gz 2024-08-25T21:09:38.639 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.22737.log 2024-08-25T21:09:38.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113824.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113824.log.gz 2024-08-25T21:09:38.647 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33976.log 2024-08-25T21:09:38.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.22737.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84321.log 2024-08-25T21:09:38.647 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.22737.log.gz 2024-08-25T21:09:38.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33976.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33976.log.gz 2024-08-25T21:09:38.647 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112079.log 2024-08-25T21:09:38.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84321.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84321.log.gz 2024-08-25T21:09:38.647 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62306.log 2024-08-25T21:09:38.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112079.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112079.log.gz 2024-08-25T21:09:38.647 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22959.log 2024-08-25T21:09:38.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62306.log.gz 2024-08-25T21:09:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.142341.log 2024-08-25T21:09:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22959.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22959.log.gz 2024-08-25T21:09:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69129.log 2024-08-25T21:09:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.142341.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.142341.log.gz 2024-08-25T21:09:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34282.log 2024-08-25T21:09:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.69129.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69129.log.gz 2024-08-25T21:09:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28355.log 2024-08-25T21:09:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34282.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34282.log.gz 2024-08-25T21:09:38.648 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45950.log 2024-08-25T21:09:38.650 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28355.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50479.log 2024-08-25T21:09:38.650 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28355.log.gz 2024-08-25T21:09:38.650 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.45950.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31295.log 2024-08-25T21:09:38.650 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45950.log.gz 2024-08-25T21:09:38.650 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50479.log.gz 2024-08-25T21:09:38.650 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120000.log 2024-08-25T21:09:38.650 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31295.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31295.log.gz 2024-08-25T21:09:38.651 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24284.log 2024-08-25T21:09:38.651 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120000.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120000.log.gz 2024-08-25T21:09:38.652 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49075.log 2024-08-25T21:09:38.652 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24284.log.gz 2024-08-25T21:09:38.652 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106957.log 2024-08-25T21:09:38.653 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49075.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49075.log.gz 2024-08-25T21:09:38.653 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118442.log 2024-08-25T21:09:38.653 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106957.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106957.log.gz 2024-08-25T21:09:38.653 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26462.log 2024-08-25T21:09:38.654 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118442.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118442.log.gz 2024-08-25T21:09:38.654 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87158.log 2024-08-25T21:09:38.655 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26462.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26462.log.gz 2024-08-25T21:09:38.655 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103861.log 2024-08-25T21:09:38.655 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87158.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87158.log.gz 2024-08-25T21:09:38.655 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57060.log 2024-08-25T21:09:38.656 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103861.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107302.log 2024-08-25T21:09:38.656 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103861.log.gz 2024-08-25T21:09:38.657 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57060.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57060.log.gz 2024-08-25T21:09:38.657 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84471.log 2024-08-25T21:09:38.658 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107302.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107302.log.gz 2024-08-25T21:09:38.658 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127892.log 2024-08-25T21:09:38.659 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84471.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84471.log.gz 2024-08-25T21:09:38.659 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74086.log 2024-08-25T21:09:38.659 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.127892.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127892.log.gz 2024-08-25T21:09:38.659 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31610.log 2024-08-25T21:09:38.660 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74086.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74086.log.gz 2024-08-25T21:09:38.660 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42315.log 2024-08-25T21:09:38.661 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31610.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31610.log.gz 2024-08-25T21:09:38.661 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84365.log 2024-08-25T21:09:38.662 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42315.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42315.log.gz 2024-08-25T21:09:38.662 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132901.log 2024-08-25T21:09:38.662 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84365.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84365.log.gz 2024-08-25T21:09:38.663 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117530.log 2024-08-25T21:09:38.663 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132901.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132901.log.gz 2024-08-25T21:09:38.663 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55817.log 2024-08-25T21:09:38.664 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.117530.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117530.log.gz 2024-08-25T21:09:38.664 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131810.log 2024-08-25T21:09:38.665 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55817.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55817.log.gz 2024-08-25T21:09:38.665 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99768.log 2024-08-25T21:09:38.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131810.log.gz 2024-08-25T21:09:38.666 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56788.log 2024-08-25T21:09:38.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99768.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99768.log.gz 2024-08-25T21:09:38.667 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89758.log 2024-08-25T21:09:38.667 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56788.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56788.log.gz 2024-08-25T21:09:38.667 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59627.log 2024-08-25T21:09:38.668 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89758.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89758.log.gz 2024-08-25T21:09:38.668 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105132.log 2024-08-25T21:09:38.669 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59627.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59627.log.gz 2024-08-25T21:09:38.669 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.20869.log 2024-08-25T21:09:38.670 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105132.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105132.log.gz 2024-08-25T21:09:38.670 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139188.log 2024-08-25T21:09:38.670 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.20869.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.20869.log.gz 2024-08-25T21:09:38.670 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.22426.log 2024-08-25T21:09:38.671 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139188.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.admin.139188.log.gz /var/log/ceph/cluster1-client.mirror.24106.log 2024-08-25T21:09:38.671 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:38.672 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.22426.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58224.log 2024-08-25T21:09:38.673 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24106.log.gz 2024-08-25T21:09:38.673 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60431.log 2024-08-25T21:09:38.674 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58224.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58224.log.gz 2024-08-25T21:09:38.674 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87952.log 2024-08-25T21:09:38.675 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60431.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60431.log.gz 2024-08-25T21:09:38.675 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.21764.log 2024-08-25T21:09:38.676 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87952.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87952.log.gz 2024-08-25T21:09:38.676 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46588.log 2024-08-25T21:09:38.676 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.21764.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.21764.log.gz 2024-08-25T21:09:38.677 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44124.log 2024-08-25T21:09:38.677 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46588.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46588.log.gz 2024-08-25T21:09:38.677 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46333.log 2024-08-25T21:09:38.678 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44124.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44124.log.gz 2024-08-25T21:09:38.678 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125387.log 2024-08-25T21:09:38.678 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46333.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46333.log.gz 2024-08-25T21:09:38.679 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57788.log 2024-08-25T21:09:38.679 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125387.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125387.log.gz 2024-08-25T21:09:38.679 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53376.log 2024-08-25T21:09:38.680 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57788.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122890.log 2024-08-25T21:09:38.680 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57788.log.gz 2024-08-25T21:09:38.681 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53376.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53376.log.gz 2024-08-25T21:09:38.681 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87050.log 2024-08-25T21:09:38.682 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122890.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122890.log.gz 2024-08-25T21:09:38.682 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59969.log 2024-08-25T21:09:38.682 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87050.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87050.log.gzgzip 2024-08-25T21:09:38.683 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83858.log 2024-08-25T21:09:38.683 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59969.log.gz 2024-08-25T21:09:38.683 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38617.log 2024-08-25T21:09:38.685 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83858.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83858.log.gz 2024-08-25T21:09:38.685 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78429.log 2024-08-25T21:09:38.685 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38617.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38617.log.gz 2024-08-25T21:09:38.685 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89107.log 2024-08-25T21:09:38.686 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78429.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78429.log.gz 2024-08-25T21:09:38.686 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64740.log 2024-08-25T21:09:38.686 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89107.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89107.log.gz 2024-08-25T21:09:38.686 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119237.log 2024-08-25T21:09:38.687 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64740.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134712.log 0.0% 2024-08-25T21:09:38.687 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.64740.log.gz 2024-08-25T21:09:38.688 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119237.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119237.log.gz 2024-08-25T21:09:38.688 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41895.log 2024-08-25T21:09:38.688 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134712.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134712.log.gz 2024-08-25T21:09:38.689 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72550.log 2024-08-25T21:09:38.689 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41895.log.gz 2024-08-25T21:09:38.689 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85873.log 2024-08-25T21:09:38.690 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72550.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72550.log.gz 2024-08-25T21:09:38.690 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60711.log 2024-08-25T21:09:38.691 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85873.log.gz 2024-08-25T21:09:38.691 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94923.log 2024-08-25T21:09:38.692 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60711.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60711.log.gz 2024-08-25T21:09:38.692 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.130341.log 2024-08-25T21:09:38.692 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94923.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94923.log.gz 2024-08-25T21:09:38.693 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48591.log 2024-08-25T21:09:38.693 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.130341.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.130341.log.gz 2024-08-25T21:09:38.693 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114244.log 2024-08-25T21:09:38.694 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48591.log.gz 2024-08-25T21:09:38.694 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78687.log 2024-08-25T21:09:38.695 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114244.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114244.log.gz 2024-08-25T21:09:38.695 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89588.log 2024-08-25T21:09:38.695 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78687.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78687.log.gz 2024-08-25T21:09:38.695 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128023.log 2024-08-25T21:09:38.696 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89588.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89588.log.gz 2024-08-25T21:09:38.696 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113569.log 2024-08-25T21:09:38.696 INFO:teuthology.orchestra.run.smithi121.stderr: 89.0% -- replaced with /var/log/ceph/cluster1-client.mirror.22426.log.gz 2024-08-25T21:09:38.697 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25316.log 2024-08-25T21:09:38.697 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.128023.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128023.log.gz 2024-08-25T21:09:38.698 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113569.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77366.log 2024-08-25T21:09:38.698 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113569.log.gz 2024-08-25T21:09:38.699 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25316.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25316.log.gz 2024-08-25T21:09:38.699 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67316.log 2024-08-25T21:09:38.699 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77366.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75702.log 2024-08-25T21:09:38.699 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77366.log.gz 2024-08-25T21:09:38.700 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67316.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67316.log.gz 2024-08-25T21:09:38.700 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115364.log 2024-08-25T21:09:38.701 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75702.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75702.log.gz 2024-08-25T21:09:38.701 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75123.log 2024-08-25T21:09:38.701 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115364.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96558.log 2024-08-25T21:09:38.702 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115364.log.gz 2024-08-25T21:09:38.702 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75123.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75123.log.gz 2024-08-25T21:09:38.702 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76692.log 2024-08-25T21:09:38.703 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96558.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96558.log.gz 2024-08-25T21:09:38.703 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71810.log 2024-08-25T21:09:38.704 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76692.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104153.log 2024-08-25T21:09:38.704 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76692.log.gz 2024-08-25T21:09:38.705 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71810.log.gz 2024-08-25T21:09:38.705 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30897.log 2024-08-25T21:09:38.706 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.104153.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104153.log.gzgzip 2024-08-25T21:09:38.706 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35122.log 2024-08-25T21:09:38.707 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.30897.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30897.log.gz 2024-08-25T21:09:38.707 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44345.log 2024-08-25T21:09:38.708 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35122.log.gz 2024-08-25T21:09:38.708 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58198.log 2024-08-25T21:09:38.708 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44345.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44345.log.gz 2024-08-25T21:09:38.708 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58493.log 2024-08-25T21:09:38.709 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58198.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58198.log.gz 2024-08-25T21:09:38.709 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62871.log 2024-08-25T21:09:38.710 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.58493.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58493.log.gz 2024-08-25T21:09:38.710 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32466.log 2024-08-25T21:09:38.711 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62871.log.gz 2024-08-25T21:09:38.711 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27009.log 2024-08-25T21:09:38.712 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32466.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32466.log.gz 2024-08-25T21:09:38.712 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124711.log 2024-08-25T21:09:38.712 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27009.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.27009.log.gz 2024-08-25T21:09:38.712 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49576.log 2024-08-25T21:09:38.713 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124711.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124711.log.gz 2024-08-25T21:09:38.713 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107633.log 2024-08-25T21:09:38.714 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49576.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49576.log.gz 2024-08-25T21:09:38.714 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65078.log 2024-08-25T21:09:38.715 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.107633.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107633.log.gz 2024-08-25T21:09:38.715 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98025.log 2024-08-25T21:09:38.716 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65078.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65078.log.gz 2024-08-25T21:09:38.716 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104154.log 2024-08-25T21:09:38.716 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98025.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98025.log.gz 2024-08-25T21:09:38.717 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46272.log 2024-08-25T21:09:38.717 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.104154.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104154.log.gz 2024-08-25T21:09:38.717 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139189.log 2024-08-25T21:09:38.718 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46272.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46272.log.gz 2024-08-25T21:09:38.718 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121562.log 2024-08-25T21:09:38.719 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139189.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139189.log.gz 2024-08-25T21:09:38.719 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.21929.log 2024-08-25T21:09:38.720 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121562.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121562.log.gz 2024-08-25T21:09:38.720 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.17066.log 2024-08-25T21:09:38.720 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.21929.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.21929.log.gz 2024-08-25T21:09:38.720 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66136.log 2024-08-25T21:09:38.721 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.17066.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.17066.log.gz 2024-08-25T21:09:38.721 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25107.log 2024-08-25T21:09:38.722 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66136.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66136.log.gz 2024-08-25T21:09:38.722 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120391.log 2024-08-25T21:09:38.723 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25107.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25107.log.gz 2024-08-25T21:09:38.723 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53251.log 2024-08-25T21:09:38.724 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120391.log.gz 2024-08-25T21:09:38.724 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64507.log 2024-08-25T21:09:38.725 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53251.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53251.log.gz 2024-08-25T21:09:38.725 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107020.log 2024-08-25T21:09:38.725 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64507.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118910.log 2024-08-25T21:09:38.726 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107020.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107020.log.gz 2024-08-25T21:09:38.726 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101849.log 2024-08-25T21:09:38.727 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118910.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118910.log.gz 2024-08-25T21:09:38.727 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54960.log 2024-08-25T21:09:38.728 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101849.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64507.log.gz 2024-08-25T21:09:38.728 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101849.log.gz 2024-08-25T21:09:38.728 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36482.log 2024-08-25T21:09:38.729 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54960.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.54960.log.gz 2024-08-25T21:09:38.729 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.32338.log 2024-08-25T21:09:38.729 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36482.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36482.log.gz 2024-08-25T21:09:38.729 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89800.log 2024-08-25T21:09:38.730 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.32338.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32338.log.gz 2024-08-25T21:09:38.730 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135074.log 2024-08-25T21:09:38.731 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89800.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89800.log.gz 2024-08-25T21:09:38.731 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135703.log 2024-08-25T21:09:38.732 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135074.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33937.log 2024-08-25T21:09:38.732 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135074.log.gz 2024-08-25T21:09:38.732 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135703.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135703.log.gz 2024-08-25T21:09:38.732 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34028.log 2024-08-25T21:09:38.733 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33937.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68051.log 2024-08-25T21:09:38.733 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33937.log.gz 2024-08-25T21:09:38.734 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34028.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34028.log.gz 2024-08-25T21:09:38.734 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47452.log 2024-08-25T21:09:38.735 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68051.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68051.log.gz 2024-08-25T21:09:38.735 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103491.log 2024-08-25T21:09:38.736 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47452.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47452.log.gz 2024-08-25T21:09:38.736 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133002.log 2024-08-25T21:09:38.737 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103491.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103491.log.gz 2024-08-25T21:09:38.737 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51067.log 2024-08-25T21:09:38.738 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133002.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133002.log.gz 2024-08-25T21:09:38.738 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95623.log 2024-08-25T21:09:38.738 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51067.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51067.log.gz 2024-08-25T21:09:38.739 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136039.log 2024-08-25T21:09:38.740 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95623.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95623.log.gz 2024-08-25T21:09:38.740 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69192.log 2024-08-25T21:09:38.740 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136039.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136039.log.gz 2024-08-25T21:09:38.740 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45506.log 2024-08-25T21:09:38.741 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69192.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69192.log.gz 2024-08-25T21:09:38.741 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53587.log 2024-08-25T21:09:38.741 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45506.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45506.log.gz 2024-08-25T21:09:38.741 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101617.log 2024-08-25T21:09:38.742 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53587.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53587.log.gz 2024-08-25T21:09:38.742 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121166.log 2024-08-25T21:09:38.743 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101617.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101617.log.gz 2024-08-25T21:09:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132085.log 2024-08-25T21:09:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121166.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121166.log.gz 2024-08-25T21:09:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22268.log 2024-08-25T21:09:38.744 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132085.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132085.log.gz 2024-08-25T21:09:38.745 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36644.log 2024-08-25T21:09:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22268.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22268.log.gz 2024-08-25T21:09:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28982.log 2024-08-25T21:09:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36644.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36644.log.gz 2024-08-25T21:09:38.746 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127253.log 2024-08-25T21:09:38.747 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28982.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28982.log.gz 2024-08-25T21:09:38.747 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74928.log 2024-08-25T21:09:38.748 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127253.log.gz 2024-08-25T21:09:38.748 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93511.log 2024-08-25T21:09:38.748 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74928.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74928.log.gz 2024-08-25T21:09:38.748 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113400.log 2024-08-25T21:09:38.749 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93511.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93511.log.gz 2024-08-25T21:09:38.749 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28779.log 2024-08-25T21:09:38.750 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113400.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113400.log.gz 2024-08-25T21:09:38.750 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117899.log 2024-08-25T21:09:38.751 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28779.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28779.log.gz 2024-08-25T21:09:38.751 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28395.log 2024-08-25T21:09:38.752 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.117899.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117899.log.gz 2024-08-25T21:09:38.752 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79572.log 2024-08-25T21:09:38.753 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28395.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.28395.log.gz 2024-08-25T21:09:38.753 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125283.log 2024-08-25T21:09:38.753 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79572.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79572.log.gz 2024-08-25T21:09:38.754 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116350.log 2024-08-25T21:09:38.754 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125283.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125283.log.gz 2024-08-25T21:09:38.755 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63524.log 2024-08-25T21:09:38.756 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.116350.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.116350.log.gz 2024-08-25T21:09:38.756 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101533.log 2024-08-25T21:09:38.756 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63524.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63524.log.gz 2024-08-25T21:09:38.757 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48371.log 2024-08-25T21:09:38.758 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101533.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101533.log.gz 2024-08-25T21:09:38.758 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79573.log 2024-08-25T21:09:38.759 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48371.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48371.log.gz 2024-08-25T21:09:38.759 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100108.log 2024-08-25T21:09:38.759 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79573.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79573.log.gz 2024-08-25T21:09:38.760 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117177.log 2024-08-25T21:09:38.760 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100108.log.gz 2024-08-25T21:09:38.761 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100673.log 2024-08-25T21:09:38.761 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117177.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117177.log.gz 2024-08-25T21:09:38.762 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44991.log 2024-08-25T21:09:38.762 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100673.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100673.log.gz 2024-08-25T21:09:38.763 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125494.log 2024-08-25T21:09:38.763 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44991.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44991.log.gz 2024-08-25T21:09:38.764 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.23044.log 2024-08-25T21:09:38.764 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125494.log.gz 2024-08-25T21:09:38.764 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95969.log 2024-08-25T21:09:38.765 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.23044.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.23044.log.gz 2024-08-25T21:09:38.766 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51447.log 2024-08-25T21:09:38.766 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95969.log.gz 2024-08-25T21:09:38.767 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24747.log 2024-08-25T21:09:38.767 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51447.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51447.log.gz 2024-08-25T21:09:38.767 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100589.log 2024-08-25T21:09:38.768 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24747.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24747.log.gz 2024-08-25T21:09:38.768 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89571.log 2024-08-25T21:09:38.769 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100589.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100589.log.gz 2024-08-25T21:09:38.769 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93245.log 2024-08-25T21:09:38.770 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89571.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89571.log.gz 2024-08-25T21:09:38.770 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74631.log 2024-08-25T21:09:38.771 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93245.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93245.log.gz 2024-08-25T21:09:38.771 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35614.log 2024-08-25T21:09:38.772 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74631.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74631.log.gz 2024-08-25T21:09:38.772 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122800.log 2024-08-25T21:09:38.773 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.35614.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35614.log.gz 2024-08-25T21:09:38.773 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60920.log 2024-08-25T21:09:38.774 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122800.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122800.log.gz 2024-08-25T21:09:38.774 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110884.log 2024-08-25T21:09:38.775 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60920.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60920.log.gz 2024-08-25T21:09:38.775 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113593.log 2024-08-25T21:09:38.776 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110884.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110884.log.gz 2024-08-25T21:09:38.776 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39627.log 2024-08-25T21:09:38.777 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113593.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113593.log.gz 2024-08-25T21:09:38.777 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56436.log 2024-08-25T21:09:38.778 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39627.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39627.log.gz 2024-08-25T21:09:38.778 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108502.log 2024-08-25T21:09:38.779 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56436.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56436.log.gz 2024-08-25T21:09:38.779 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122931.log 2024-08-25T21:09:38.780 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108502.log.gz 2024-08-25T21:09:38.780 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119913.log 2024-08-25T21:09:38.780 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122931.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122931.log.gz 2024-08-25T21:09:38.781 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85518.log 2024-08-25T21:09:38.781 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119913.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119913.log.gz 2024-08-25T21:09:38.782 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137472.log 2024-08-25T21:09:38.782 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85518.log.gz 2024-08-25T21:09:38.782 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48652.log 2024-08-25T21:09:38.783 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137472.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137472.log.gz 2024-08-25T21:09:38.784 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34739.log 2024-08-25T21:09:38.784 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48652.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48652.log.gz 2024-08-25T21:09:38.784 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42798.log 2024-08-25T21:09:38.785 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34739.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34739.log.gz 2024-08-25T21:09:38.785 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75112.log 2024-08-25T21:09:38.786 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42798.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42798.log.gz 2024-08-25T21:09:38.786 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53883.log 2024-08-25T21:09:38.787 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75112.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75112.log.gz 2024-08-25T21:09:38.787 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88523.log 2024-08-25T21:09:38.788 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53883.log.gz 2024-08-25T21:09:38.788 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38914.log 2024-08-25T21:09:38.789 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88523.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88523.log.gz 2024-08-25T21:09:38.789 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54006.log 2024-08-25T21:09:38.790 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38914.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38914.log.gz 2024-08-25T21:09:38.790 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34780.log 2024-08-25T21:09:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54006.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54006.log.gz 2024-08-25T21:09:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62934.log 2024-08-25T21:09:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.34780.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34780.log.gz 2024-08-25T21:09:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138900.log 2024-08-25T21:09:38.793 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62934.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62934.log.gz 2024-08-25T21:09:38.793 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40087.log 2024-08-25T21:09:38.794 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138900.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138900.log.gz 2024-08-25T21:09:38.794 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105513.log 2024-08-25T21:09:38.795 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40087.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40087.log.gz 2024-08-25T21:09:38.795 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23776.log 2024-08-25T21:09:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105513.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105513.log.gz 2024-08-25T21:09:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68124.log 2024-08-25T21:09:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23776.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23776.log.gz 2024-08-25T21:09:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103562.log 2024-08-25T21:09:38.798 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68124.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68124.log.gz 2024-08-25T21:09:38.798 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117439.log 2024-08-25T21:09:38.799 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103562.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103562.log.gz 2024-08-25T21:09:38.799 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48132.log 2024-08-25T21:09:38.800 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117439.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117439.log.gz 2024-08-25T21:09:38.800 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68020.log 2024-08-25T21:09:38.801 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48132.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48132.log.gz 2024-08-25T21:09:38.801 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85979.log 2024-08-25T21:09:38.802 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68020.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68020.log.gz 2024-08-25T21:09:38.802 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125935.log 2024-08-25T21:09:38.803 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85979.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85979.log.gz 2024-08-25T21:09:38.803 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101852.log 2024-08-25T21:09:38.804 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125935.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125935.log.gz 2024-08-25T21:09:38.804 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.119595.log 2024-08-25T21:09:38.805 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101852.log.gz 2024-08-25T21:09:38.805 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108292.log 2024-08-25T21:09:38.806 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.119595.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.119595.log.gz 2024-08-25T21:09:38.806 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52788.log 2024-08-25T21:09:38.807 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108292.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108292.log.gz 2024-08-25T21:09:38.807 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113526.log 2024-08-25T21:09:38.807 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52788.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52788.log.gz 2024-08-25T21:09:38.808 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95714.log 2024-08-25T21:09:38.808 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113526.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113526.log.gz 2024-08-25T21:09:38.809 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61025.log 2024-08-25T21:09:38.810 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95714.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95714.log.gz 2024-08-25T21:09:38.810 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127164.log 2024-08-25T21:09:38.811 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61025.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61025.log.gz 2024-08-25T21:09:38.811 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78032.log 2024-08-25T21:09:38.811 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127164.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127164.log.gz 2024-08-25T21:09:38.812 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110796.log 2024-08-25T21:09:38.812 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78032.log.gz 2024-08-25T21:09:38.813 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138922.log 2024-08-25T21:09:38.813 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110796.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110796.log.gz 2024-08-25T21:09:38.814 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85182.log 2024-08-25T21:09:38.814 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138922.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138922.log.gz 2024-08-25T21:09:38.814 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115848.log 2024-08-25T21:09:38.815 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85182.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85182.log.gz 2024-08-25T21:09:38.816 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124337.log 2024-08-25T21:09:38.816 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115848.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115848.log.gz 2024-08-25T21:09:38.817 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123454.log 2024-08-25T21:09:38.817 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124337.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124337.log.gz 2024-08-25T21:09:38.817 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45877.log 2024-08-25T21:09:38.818 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123454.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123454.log.gz 2024-08-25T21:09:38.818 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54090.log 2024-08-25T21:09:38.819 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.45877.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45877.log.gz 2024-08-25T21:09:38.819 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137256.log 2024-08-25T21:09:38.820 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54090.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54090.log.gz 2024-08-25T21:09:38.820 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84185.log 2024-08-25T21:09:38.821 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137256.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137256.log.gz 2024-08-25T21:09:38.821 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39173.log 2024-08-25T21:09:38.822 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84185.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84185.log.gz 2024-08-25T21:09:38.822 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124865.log 2024-08-25T21:09:38.823 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39173.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39173.log.gz 2024-08-25T21:09:38.823 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48479.log 2024-08-25T21:09:38.824 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124865.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124865.log.gz 2024-08-25T21:09:38.825 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110862.log 2024-08-25T21:09:38.825 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48479.log.gz 2024-08-25T21:09:38.825 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73474.log 2024-08-25T21:09:38.826 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110862.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110862.log.gz 2024-08-25T21:09:38.826 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130493.log 2024-08-25T21:09:38.827 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73474.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73474.log.gz 2024-08-25T21:09:38.827 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108460.log 2024-08-25T21:09:38.828 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130493.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130493.log.gz 2024-08-25T21:09:38.828 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25630.log 2024-08-25T21:09:38.829 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108460.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108460.log.gz 2024-08-25T21:09:38.829 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49975.log 2024-08-25T21:09:38.830 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25630.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25630.log.gz 2024-08-25T21:09:38.830 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113697.log 2024-08-25T21:09:38.831 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49975.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49975.log.gz 2024-08-25T21:09:38.831 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135973.log 2024-08-25T21:09:38.832 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113697.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113697.log.gz 2024-08-25T21:09:38.832 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76901.log 2024-08-25T21:09:38.833 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135973.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135973.log.gz 2024-08-25T21:09:38.833 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123983.log 2024-08-25T21:09:38.834 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76901.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76901.log.gz 2024-08-25T21:09:38.834 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120808.log 2024-08-25T21:09:38.835 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123983.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123983.log.gz 2024-08-25T21:09:38.835 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85078.log 2024-08-25T21:09:38.836 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120808.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120808.log.gz 2024-08-25T21:09:38.836 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64068.log 2024-08-25T21:09:38.837 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85078.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85078.log.gz 2024-08-25T21:09:38.837 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85643.log 2024-08-25T21:09:38.838 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64068.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64068.log.gz 2024-08-25T21:09:38.838 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41727.log 2024-08-25T21:09:38.839 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85643.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85643.log.gz 2024-08-25T21:09:38.839 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62578.log 2024-08-25T21:09:38.839 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41727.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41727.log.gz 2024-08-25T21:09:38.840 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48549.log 2024-08-25T21:09:38.840 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62578.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62578.log.gz 2024-08-25T21:09:38.840 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131230.log 2024-08-25T21:09:38.841 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48549.log.gz 2024-08-25T21:09:38.841 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25002.log 2024-08-25T21:09:38.842 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131230.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131230.log.gz 2024-08-25T21:09:38.842 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68234.log 2024-08-25T21:09:38.843 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25002.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25002.log.gz 2024-08-25T21:09:38.843 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104593.log 2024-08-25T21:09:38.844 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68234.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68234.log.gz 2024-08-25T21:09:38.844 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101910.log 2024-08-25T21:09:38.845 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.104593.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104593.log.gz 2024-08-25T21:09:38.845 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34713.log 2024-08-25T21:09:38.846 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101910.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101910.log.gz 2024-08-25T21:09:38.846 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88082.log 2024-08-25T21:09:38.847 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34713.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34713.log.gz 2024-08-25T21:09:38.847 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68141.log 2024-08-25T21:09:38.848 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88082.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88082.log.gz 2024-08-25T21:09:38.848 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84295.log 2024-08-25T21:09:38.849 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68141.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68141.log.gz 2024-08-25T21:09:38.849 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94765.log 2024-08-25T21:09:38.850 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84295.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84295.log.gz 2024-08-25T21:09:38.850 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83375.log 2024-08-25T21:09:38.850 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94765.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94765.log.gz 2024-08-25T21:09:38.851 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130381.log 2024-08-25T21:09:38.851 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83375.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83375.log.gz 2024-08-25T21:09:38.851 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104771.log 2024-08-25T21:09:38.852 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130381.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130381.log.gz 2024-08-25T21:09:38.853 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140455.log 2024-08-25T21:09:38.853 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.104771.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104771.log.gz 2024-08-25T21:09:38.853 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63186.log 2024-08-25T21:09:38.854 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.140455.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140455.log.gz 2024-08-25T21:09:38.854 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79680.log 2024-08-25T21:09:38.855 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63186.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63186.log.gz 2024-08-25T21:09:38.855 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77320.log 2024-08-25T21:09:38.856 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79680.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79680.log.gz 2024-08-25T21:09:38.856 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35426.log 2024-08-25T21:09:38.857 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77320.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77320.log.gz 2024-08-25T21:09:38.857 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98633.log 2024-08-25T21:09:38.858 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35426.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35426.log.gz 2024-08-25T21:09:38.858 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40507.log 2024-08-25T21:09:38.859 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98633.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98633.log.gz 2024-08-25T21:09:38.859 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103428.log 2024-08-25T21:09:38.860 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40507.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40507.log.gz 2024-08-25T21:09:38.860 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102289.log 2024-08-25T21:09:38.861 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103428.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103428.log.gz 2024-08-25T21:09:38.861 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53502.log 2024-08-25T21:09:38.862 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102289.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102289.log.gz 2024-08-25T21:09:38.862 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106174.log 2024-08-25T21:09:38.863 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53502.log.gz 2024-08-25T21:09:38.863 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121040.log 2024-08-25T21:09:38.864 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106174.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106174.log.gz 2024-08-25T21:09:38.864 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68810.log 2024-08-25T21:09:38.865 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121040.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121040.log.gz 2024-08-25T21:09:38.865 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44252.log 2024-08-25T21:09:38.866 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68810.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68810.log.gz 2024-08-25T21:09:38.866 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116177.log 2024-08-25T21:09:38.867 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44252.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44252.log.gz 2024-08-25T21:09:38.867 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30076.log 2024-08-25T21:09:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116177.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116177.log.gz 2024-08-25T21:09:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82062.log 2024-08-25T21:09:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30076.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30076.log.gz 2024-08-25T21:09:38.869 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101492.log 2024-08-25T21:09:38.869 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82062.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82062.log.gz 2024-08-25T21:09:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102056.log 2024-08-25T21:09:38.871 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101492.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101492.log.gz 2024-08-25T21:09:38.871 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46842.log 2024-08-25T21:09:38.871 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102056.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102056.log.gz 2024-08-25T21:09:38.872 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.21671.log 2024-08-25T21:09:38.872 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46842.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46842.log.gz 2024-08-25T21:09:38.872 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35661.log 2024-08-25T21:09:38.873 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.21671.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.21671.log.gz 2024-08-25T21:09:38.873 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134864.log 2024-08-25T21:09:38.874 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35661.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35661.log.gz 2024-08-25T21:09:38.874 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93958.log 2024-08-25T21:09:38.875 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134864.log.gz 2024-08-25T21:09:38.875 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123455.log 2024-08-25T21:09:38.876 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.93958.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93958.log.gz 2024-08-25T21:09:38.876 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51908.log 2024-08-25T21:09:38.877 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123455.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123455.log.gz 2024-08-25T21:09:38.877 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31873.log 2024-08-25T21:09:38.878 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51908.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51908.log.gz 2024-08-25T21:09:38.878 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64364.log 2024-08-25T21:09:38.879 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31873.log.gz 2024-08-25T21:09:38.879 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134460.log 2024-08-25T21:09:38.880 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64364.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64364.log.gz 2024-08-25T21:09:38.880 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96767.log 2024-08-25T21:09:38.880 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134460.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134460.log.gz 2024-08-25T21:09:38.881 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46930.log 2024-08-25T21:09:38.882 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96767.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96767.log.gz 2024-08-25T21:09:38.882 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135553.log 2024-08-25T21:09:38.882 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46930.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46930.log.gz 2024-08-25T21:09:38.883 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.33436.log 2024-08-25T21:09:38.884 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135553.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135553.log.gz 2024-08-25T21:09:38.884 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121884.log 2024-08-25T21:09:38.884 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.33436.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.33436.log.gz 2024-08-25T21:09:38.885 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88146.log 2024-08-25T21:09:38.885 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121884.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121884.log.gz 2024-08-25T21:09:38.886 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107564.log 2024-08-25T21:09:38.887 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88146.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88146.log.gz 2024-08-25T21:09:38.887 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77278.log 2024-08-25T21:09:38.887 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107564.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107564.log.gz 2024-08-25T21:09:38.888 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123768.log 2024-08-25T21:09:38.888 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77278.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77278.log.gz 2024-08-25T21:09:38.889 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30946.log 2024-08-25T21:09:38.889 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123768.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123768.log.gz 2024-08-25T21:09:38.890 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70646.log 2024-08-25T21:09:38.890 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30946.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30946.log.gz 2024-08-25T21:09:38.891 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129026.log 2024-08-25T21:09:38.891 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70646.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70646.log.gz 2024-08-25T21:09:38.891 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103116.log 2024-08-25T21:09:38.892 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.129026.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129026.log.gz 2024-08-25T21:09:38.892 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66337.log 2024-08-25T21:09:38.893 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103116.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103116.log.gz 2024-08-25T21:09:38.893 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28292.log 2024-08-25T21:09:38.894 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66337.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66337.log.gz 2024-08-25T21:09:38.894 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127269.log 2024-08-25T21:09:38.895 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28292.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28292.log.gz 2024-08-25T21:09:38.895 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46064.log 2024-08-25T21:09:38.896 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127269.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.127269.log.gz -5 2024-08-25T21:09:38.896 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster2-client.admin.57144.log 2024-08-25T21:09:38.896 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46064.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.46064.log.gz -5 2024-08-25T21:09:38.897 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.27599.log 2024-08-25T21:09:38.897 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57144.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57144.log.gz 2024-08-25T21:09:38.897 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142060.log 2024-08-25T21:09:38.898 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27599.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27599.log.gz 2024-08-25T21:09:38.898 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86357.log 2024-08-25T21:09:38.899 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.142060.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142060.log.gz 2024-08-25T21:09:38.899 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97184.log 2024-08-25T21:09:38.900 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86357.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86357.log.gz 2024-08-25T21:09:38.900 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86903.log 2024-08-25T21:09:38.901 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97184.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97184.log.gz 2024-08-25T21:09:38.901 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81378.log 2024-08-25T21:09:38.901 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86903.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.86903.log.gz -5 2024-08-25T21:09:38.901 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.40174.log 2024-08-25T21:09:38.902 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81378.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81378.log.gz 2024-08-25T21:09:38.902 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41305.log 2024-08-25T21:09:38.903 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40174.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40174.log.gz 2024-08-25T21:09:38.903 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39123.log 2024-08-25T21:09:38.904 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41305.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41305.log.gz 2024-08-25T21:09:38.904 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90489.log 2024-08-25T21:09:38.904 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39123.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39123.log.gz 2024-08-25T21:09:38.904 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62703.log 2024-08-25T21:09:38.905 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90489.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90489.log.gz 2024-08-25T21:09:38.905 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141973.log 2024-08-25T21:09:38.906 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62703.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62703.log.gz 2024-08-25T21:09:38.906 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140153.log 2024-08-25T21:09:38.907 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141973.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141973.log.gz 2024-08-25T21:09:38.907 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84681.log 2024-08-25T21:09:38.907 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.140153.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.admin.140153.log.gz /var/log/ceph/cluster1-client.mirror.124023.log 2024-08-25T21:09:38.907 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:38.908 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84681.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84681.log.gz 2024-08-25T21:09:38.908 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76691.log 2024-08-25T21:09:38.909 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124023.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124023.log.gz 2024-08-25T21:09:38.909 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106033.log 2024-08-25T21:09:38.910 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76691.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128663.log 2024-08-25T21:09:38.910 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76691.log.gz 2024-08-25T21:09:38.910 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106033.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106033.log.gz 2024-08-25T21:09:38.911 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118779.log 2024-08-25T21:09:38.911 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128663.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128663.log.gz 2024-08-25T21:09:38.911 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.142927.log 2024-08-25T21:09:38.912 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118779.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92434.log 2024-08-25T21:09:38.912 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118779.log.gz 2024-08-25T21:09:38.913 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.142927.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.142927.log.gz 2024-08-25T21:09:38.913 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94072.log 2024-08-25T21:09:38.914 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92434.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92434.log.gz 2024-08-25T21:09:38.914 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47864.log 2024-08-25T21:09:38.914 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94072.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94072.log.gz 2024-08-25T21:09:38.914 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58513.log 2024-08-25T21:09:38.915 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47864.log.gz 2024-08-25T21:09:38.915 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58768.log 2024-08-25T21:09:38.916 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58513.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58513.log.gz 2024-08-25T21:09:38.916 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45673.log 2024-08-25T21:09:38.916 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58768.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58768.log.gz 2024-08-25T21:09:38.917 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101535.log 2024-08-25T21:09:38.917 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45673.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45673.log.gz 2024-08-25T21:09:38.917 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60501.log 2024-08-25T21:09:38.918 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101535.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101535.log.gz 2024-08-25T21:09:38.918 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88987.log 2024-08-25T21:09:38.919 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60501.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60501.log.gz 2024-08-25T21:09:38.919 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84683.log 2024-08-25T21:09:38.920 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88987.log.gz 2024-08-25T21:09:38.920 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73324.log 2024-08-25T21:09:38.920 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84683.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84683.log.gz 2024-08-25T21:09:38.920 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63146.log 2024-08-25T21:09:38.921 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73324.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73324.log.gz 2024-08-25T21:09:38.921 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43038.log 2024-08-25T21:09:38.922 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63146.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63146.log.gz 2024-08-25T21:09:38.922 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130993.log 2024-08-25T21:09:38.923 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43038.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43038.log.gz 2024-08-25T21:09:38.923 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47933.log 2024-08-25T21:09:38.924 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.130993.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130993.log.gz 2024-08-25T21:09:38.924 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42148.log 2024-08-25T21:09:38.925 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.47933.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47933.log.gz 2024-08-25T21:09:38.925 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93527.log 2024-08-25T21:09:38.925 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42148.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42148.log.gz 2024-08-25T21:09:38.925 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62640.log 2024-08-25T21:09:38.926 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93527.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93527.log.gz 2024-08-25T21:09:38.926 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76478.log 2024-08-25T21:09:38.927 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62640.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62640.log.gz 2024-08-25T21:09:38.927 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88083.log 2024-08-25T21:09:38.928 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76478.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76478.log.gz 2024-08-25T21:09:38.928 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112332.log 2024-08-25T21:09:38.928 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88083.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88083.log.gz 2024-08-25T21:09:38.928 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142087.log 2024-08-25T21:09:38.929 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112332.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112332.log.gz 2024-08-25T21:09:38.929 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86547.log 2024-08-25T21:09:38.930 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.142087.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142087.log.gz 2024-08-25T21:09:38.930 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43630.log 2024-08-25T21:09:38.931 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86547.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86547.log.gz 2024-08-25T21:09:38.931 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132358.log 2024-08-25T21:09:38.931 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43630.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43630.log.gzgzip 2024-08-25T21:09:38.931 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35592.log 2024-08-25T21:09:38.932 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132358.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.132358.log.gz -5 2024-08-25T21:09:38.932 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.111282.log 2024-08-25T21:09:38.933 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35592.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35592.log.gz 2024-08-25T21:09:38.933 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104980.log 2024-08-25T21:09:38.933 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111282.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111282.log.gz 2024-08-25T21:09:38.934 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36335.log 2024-08-25T21:09:38.934 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104980.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104980.log.gz 2024-08-25T21:09:38.934 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139099.log 2024-08-25T21:09:38.935 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36335.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36335.log.gz 2024-08-25T21:09:38.935 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62786.log 2024-08-25T21:09:38.936 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139099.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139099.log.gz 2024-08-25T21:09:38.936 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115277.log 2024-08-25T21:09:38.937 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62786.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62786.log.gz 2024-08-25T21:09:38.937 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101536.log 2024-08-25T21:09:38.938 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115277.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115277.log.gz 2024-08-25T21:09:38.938 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52621.log 2024-08-25T21:09:38.938 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101536.log.gz 2024-08-25T21:09:38.939 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75220.log 2024-08-25T21:09:38.939 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52621.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52621.log.gz 2024-08-25T21:09:38.939 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40637.log 2024-08-25T21:09:38.940 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75220.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75220.log.gz 2024-08-25T21:09:38.940 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123582.log 2024-08-25T21:09:38.941 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40637.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40637.log.gz 2024-08-25T21:09:38.941 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37483.log 2024-08-25T21:09:38.941 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123582.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123582.log.gz -5 2024-08-25T21:09:38.941 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.25821.log 2024-08-25T21:09:38.942 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37483.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37483.log.gz 2024-08-25T21:09:38.942 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68800.log 2024-08-25T21:09:38.943 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25821.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25821.log.gz 2024-08-25T21:09:38.943 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28712.log 2024-08-25T21:09:38.944 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68800.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68800.log.gz 2024-08-25T21:09:38.944 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109569.log 2024-08-25T21:09:38.945 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28712.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28712.log.gz 2024-08-25T21:09:38.945 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25210.log 2024-08-25T21:09:38.945 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109569.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109569.log.gz 2024-08-25T21:09:38.946 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140520.log 2024-08-25T21:09:38.946 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25210.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120956.log 2024-08-25T21:09:38.946 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25210.log.gz 2024-08-25T21:09:38.947 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140520.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140520.log.gz 2024-08-25T21:09:38.947 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109592.log 2024-08-25T21:09:38.948 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120956.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120956.log.gz 2024-08-25T21:09:38.948 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77252.log 2024-08-25T21:09:38.949 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109592.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109592.log.gz 2024-08-25T21:09:38.949 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77211.log 2024-08-25T21:09:38.949 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77252.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77252.log.gz 2024-08-25T21:09:38.949 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139297.log 2024-08-25T21:09:38.950 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77211.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77211.log.gz 2024-08-25T21:09:38.950 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135704.log 2024-08-25T21:09:38.951 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139297.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139297.log.gz 2024-08-25T21:09:38.951 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51740.log 2024-08-25T21:09:38.952 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135704.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133370.log 2024-08-25T21:09:38.952 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135704.log.gz 2024-08-25T21:09:38.952 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51740.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51740.log.gz 2024-08-25T21:09:38.953 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66626.log 2024-08-25T21:09:38.953 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133370.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133370.log.gz 2024-08-25T21:09:38.953 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89314.log 2024-08-25T21:09:38.954 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66626.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66626.log.gz 2024-08-25T21:09:38.954 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129611.log 2024-08-25T21:09:38.955 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89314.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89314.log.gz 2024-08-25T21:09:38.955 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130925.log 2024-08-25T21:09:38.956 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129611.log.gz 2024-08-25T21:09:38.956 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133191.log 2024-08-25T21:09:38.957 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130925.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130925.log.gz 2024-08-25T21:09:38.957 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107897.log 2024-08-25T21:09:38.957 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133191.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133191.log.gz 2024-08-25T21:09:38.957 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131119.log 2024-08-25T21:09:38.958 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107897.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107897.log.gz 2024-08-25T21:09:38.958 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131606.log 2024-08-25T21:09:38.959 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131119.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131119.log.gz 2024-08-25T21:09:38.959 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123941.log 2024-08-25T21:09:38.960 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131606.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131606.log.gz 2024-08-25T21:09:38.960 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43164.log 2024-08-25T21:09:38.961 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123941.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123941.log.gz 2024-08-25T21:09:38.961 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28562.log 2024-08-25T21:09:38.961 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43164.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43164.log.gz 2024-08-25T21:09:38.962 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125325.log 2024-08-25T21:09:38.962 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28562.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28562.log.gz 2024-08-25T21:09:38.963 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92574.log 2024-08-25T21:09:38.963 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125325.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125325.log.gz 2024-08-25T21:09:38.963 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.32547.log 2024-08-25T21:09:38.964 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92574.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92574.log.gz 2024-08-25T21:09:38.964 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24548.log 2024-08-25T21:09:38.965 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.32547.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32547.log.gz 2024-08-25T21:09:38.965 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55729.log 2024-08-25T21:09:38.965 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24548.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24548.log.gz 2024-08-25T21:09:38.965 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46561.log 2024-08-25T21:09:38.966 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55729.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55729.log.gz 2024-08-25T21:09:38.966 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79724.log 2024-08-25T21:09:38.967 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46561.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46561.log.gz 2024-08-25T21:09:38.967 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68993.log 2024-08-25T21:09:38.968 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79724.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79724.log.gz 2024-08-25T21:09:38.968 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68952.log 2024-08-25T21:09:38.969 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68993.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68993.log.gz 2024-08-25T21:09:38.969 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76437.log 2024-08-25T21:09:38.970 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68952.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68952.log.gz 2024-08-25T21:09:38.970 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112520.log 2024-08-25T21:09:38.970 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76437.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76437.log.gz 2024-08-25T21:09:38.971 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91666.log 2024-08-25T21:09:38.972 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112520.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112520.log.gz 2024-08-25T21:09:38.972 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87103.log 2024-08-25T21:09:38.972 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91666.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91666.log.gz 2024-08-25T21:09:38.973 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65328.log 2024-08-25T21:09:38.973 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87103.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87103.log.gz 2024-08-25T21:09:38.974 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85413.log 2024-08-25T21:09:38.974 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65328.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65328.log.gz 2024-08-25T21:09:38.974 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120224.log 2024-08-25T21:09:38.975 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85413.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85413.log.gz 2024-08-25T21:09:38.975 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82357.log 2024-08-25T21:09:38.976 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120224.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120224.log.gz 2024-08-25T21:09:38.976 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48045.log 2024-08-25T21:09:38.977 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82357.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82357.log.gz 2024-08-25T21:09:38.977 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46360.log 2024-08-25T21:09:38.978 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48045.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48045.log.gz 2024-08-25T21:09:38.978 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80540.log 2024-08-25T21:09:38.979 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46360.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46360.log.gz 2024-08-25T21:09:38.979 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57388.log 2024-08-25T21:09:38.980 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80540.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80540.log.gz 2024-08-25T21:09:38.980 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.128893.log 2024-08-25T21:09:38.981 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.57388.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57388.log.gz 2024-08-25T21:09:38.981 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60544.log 2024-08-25T21:09:38.982 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.128893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.128893.log.gz 2024-08-25T21:09:38.982 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125449.log 2024-08-25T21:09:38.983 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60544.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60544.log.gz 2024-08-25T21:09:38.983 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125703.log 2024-08-25T21:09:38.983 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125449.log.gz 2024-08-25T21:09:38.984 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140133.log 2024-08-25T21:09:38.984 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125703.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125703.log.gz 2024-08-25T21:09:38.985 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25672.log 2024-08-25T21:09:38.985 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.140133.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140133.log.gz 2024-08-25T21:09:38.986 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42064.log 2024-08-25T21:09:38.986 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25672.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25672.log.gz 2024-08-25T21:09:38.987 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112691.log 2024-08-25T21:09:38.987 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42064.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42064.log.gz 2024-08-25T21:09:38.988 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97878.log 2024-08-25T21:09:38.988 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112691.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112691.log.gz 2024-08-25T21:09:38.989 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118013.log 2024-08-25T21:09:38.989 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97878.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97878.log.gz 2024-08-25T21:09:38.990 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107259.log 2024-08-25T21:09:38.990 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118013.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118013.log.gz 2024-08-25T21:09:38.991 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26842.log 2024-08-25T21:09:38.991 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107259.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107259.log.gz 2024-08-25T21:09:38.991 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94300.log 2024-08-25T21:09:38.992 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26842.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26842.log.gz 2024-08-25T21:09:38.992 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114445.log 2024-08-25T21:09:38.993 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94300.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94300.log.gz 2024-08-25T21:09:38.993 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115792.log 2024-08-25T21:09:38.994 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114445.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114445.log.gz 2024-08-25T21:09:38.994 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38197.log 2024-08-25T21:09:38.995 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115792.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115792.log.gz 2024-08-25T21:09:38.995 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142401.log 2024-08-25T21:09:38.996 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38197.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38197.log.gz 2024-08-25T21:09:38.996 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82035.log 2024-08-25T21:09:38.997 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.142401.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142401.log.gz 2024-08-25T21:09:38.997 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56675.log 2024-08-25T21:09:38.998 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82035.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82035.log.gz 2024-08-25T21:09:38.998 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95296.log 2024-08-25T21:09:38.999 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56675.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56675.log.gz 2024-08-25T21:09:38.999 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135792.log 2024-08-25T21:09:38.999 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95296.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95296.log.gz 2024-08-25T21:09:38.999 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76310.log 2024-08-25T21:09:39.000 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135792.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135792.log.gz 2024-08-25T21:09:39.000 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30495.log 2024-08-25T21:09:39.001 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76310.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76310.log.gz 2024-08-25T21:09:39.001 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50286.log 2024-08-25T21:09:39.002 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30495.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30495.log.gz 2024-08-25T21:09:39.002 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105259.log 2024-08-25T21:09:39.002 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50286.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50286.log.gz 2024-08-25T21:09:39.003 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41055.log 2024-08-25T21:09:39.003 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105259.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105259.log.gz 2024-08-25T21:09:39.003 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.33691.log 2024-08-25T21:09:39.004 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41055.log.gz 2024-08-25T21:09:39.004 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.32822.log 2024-08-25T21:09:39.005 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.33691.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.33691.log.gz 2024-08-25T21:09:39.005 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45231.log 2024-08-25T21:09:39.006 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.32822.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32822.log.gz 2024-08-25T21:09:39.006 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116137.log 2024-08-25T21:09:39.006 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45231.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45231.log.gz 2024-08-25T21:09:39.006 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53680.log 2024-08-25T21:09:39.007 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116137.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116137.log.gz 2024-08-25T21:09:39.007 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94212.log 2024-08-25T21:09:39.008 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53680.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53680.log.gz 2024-08-25T21:09:39.008 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94615.log 2024-08-25T21:09:39.009 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94212.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94212.log.gz 2024-08-25T21:09:39.009 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128712.log 2024-08-25T21:09:39.009 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94615.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94615.log.gz 2024-08-25T21:09:39.010 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43037.log 2024-08-25T21:09:39.010 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.128712.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster2-client.mirror.128712.log.gz --verbose 2024-08-25T21:09:39.010 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.admin.44803.log 2024-08-25T21:09:39.011 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43037.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43037.log.gz 2024-08-25T21:09:39.011 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104448.log 2024-08-25T21:09:39.012 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44803.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44803.log.gz 2024-08-25T21:09:39.012 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136650.log 2024-08-25T21:09:39.013 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.104448.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104448.log.gz 2024-08-25T21:09:39.013 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127668.log 2024-08-25T21:09:39.013 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136650.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136650.log.gz 2024-08-25T21:09:39.014 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97500.log 2024-08-25T21:09:39.014 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127668.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.admin.127668.log.gz -- 2024-08-25T21:09:39.014 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.admin.138788.log 2024-08-25T21:09:39.015 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97500.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121944.log 2024-08-25T21:09:39.015 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97500.log.gz 2024-08-25T21:09:39.016 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138788.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138788.log.gz 2024-08-25T21:09:39.016 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32230.log 2024-08-25T21:09:39.017 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121944.log.gz 2024-08-25T21:09:39.017 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24957.log 2024-08-25T21:09:39.017 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32230.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87470.log 0.0% 2024-08-25T21:09:39.017 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.32230.log.gz 2024-08-25T21:09:39.018 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24957.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24957.log.gz 2024-08-25T21:09:39.018 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103997.log 2024-08-25T21:09:39.019 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87470.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87470.log.gz 2024-08-25T21:09:39.019 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127562.log 2024-08-25T21:09:39.020 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.103997.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.103997.log.gz 2024-08-25T21:09:39.020 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111635.log 2024-08-25T21:09:39.020 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127562.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.admin.127562.log.gz -- 2024-08-25T21:09:39.021 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.60814.log 2024-08-25T21:09:39.021 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111635.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111635.log.gz 2024-08-25T21:09:39.021 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39457.log 2024-08-25T21:09:39.022 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60814.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60814.log.gz 2024-08-25T21:09:39.022 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129217.log 2024-08-25T21:09:39.023 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39457.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39457.log.gz 2024-08-25T21:09:39.023 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35008.log 2024-08-25T21:09:39.023 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.129217.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129217.log.gz 2024-08-25T21:09:39.024 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122195.log 2024-08-25T21:09:39.024 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35008.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35008.log.gz 2024-08-25T21:09:39.024 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102405.log 2024-08-25T21:09:39.025 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122195.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122195.log.gz 2024-08-25T21:09:39.025 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122661.log 2024-08-25T21:09:39.026 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102405.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102405.log.gz 2024-08-25T21:09:39.026 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93577.log 2024-08-25T21:09:39.027 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122661.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122661.log.gz 2024-08-25T21:09:39.027 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39584.log 2024-08-25T21:09:39.028 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93577.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93577.log.gz 2024-08-25T21:09:39.028 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90578.log 2024-08-25T21:09:39.028 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39584.log.gz 2024-08-25T21:09:39.028 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68368.log 2024-08-25T21:09:39.029 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90578.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90578.log.gzgzip 2024-08-25T21:09:39.029 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.56725.log 2024-08-25T21:09:39.030 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.68368.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.68368.log.gz 2024-08-25T21:09:39.030 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114432.log 2024-08-25T21:09:39.031 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56725.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56725.log.gz 2024-08-25T21:09:39.031 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74968.log 2024-08-25T21:09:39.031 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114432.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114432.log.gz 2024-08-25T21:09:39.032 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68142.log 2024-08-25T21:09:39.032 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74968.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74968.log.gzgzip 2024-08-25T21:09:39.032 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64866.log 2024-08-25T21:09:39.033 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68142.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68142.log.gz 2024-08-25T21:09:39.033 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120767.log 2024-08-25T21:09:39.034 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64866.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64866.log.gz 2024-08-25T21:09:39.034 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133529.log 2024-08-25T21:09:39.034 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120767.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120767.log.gz 2024-08-25T21:09:39.035 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139861.log 2024-08-25T21:09:39.035 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133529.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133529.log.gz 2024-08-25T21:09:39.035 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141315.log 2024-08-25T21:09:39.036 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.139861.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139861.log.gz 2024-08-25T21:09:39.036 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117717.log 2024-08-25T21:09:39.037 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141315.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141315.log.gz 2024-08-25T21:09:39.037 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94367.log 2024-08-25T21:09:39.038 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.117717.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117717.log.gz 2024-08-25T21:09:39.038 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64238.log 2024-08-25T21:09:39.038 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94367.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94367.log.gz 2024-08-25T21:09:39.039 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125496.log 2024-08-25T21:09:39.039 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64238.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64238.log.gz 2024-08-25T21:09:39.039 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102371.log 2024-08-25T21:09:39.040 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125496.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125496.log.gz 2024-08-25T21:09:39.040 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45344.log 2024-08-25T21:09:39.041 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102371.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102371.log.gz 2024-08-25T21:09:39.041 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38283.log 2024-08-25T21:09:39.042 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.45344.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45344.log.gz 2024-08-25T21:09:39.042 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82544.log 2024-08-25T21:09:39.042 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38283.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38283.log.gz 2024-08-25T21:09:39.043 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.95187.log 2024-08-25T21:09:39.043 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82544.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82544.log.gz 2024-08-25T21:09:39.043 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26802.log 2024-08-25T21:09:39.044 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.95187.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95187.log.gz 2024-08-25T21:09:39.044 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66474.log 2024-08-25T21:09:39.045 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26802.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124654.log 0.0% 2024-08-25T21:09:39.045 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.26802.log.gz 2024-08-25T21:09:39.045 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66474.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66474.log.gz 2024-08-25T21:09:39.046 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30007.log 2024-08-25T21:09:39.046 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124654.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124654.log.gzgzip 2024-08-25T21:09:39.046 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96705.log 2024-08-25T21:09:39.047 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30007.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30007.log.gz 2024-08-25T21:09:39.047 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99852.log 2024-08-25T21:09:39.048 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96705.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96705.log.gz 2024-08-25T21:09:39.048 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139500.log 2024-08-25T21:09:39.048 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99852.log.gz 2024-08-25T21:09:39.049 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85036.log 2024-08-25T21:09:39.049 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139500.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139500.log.gz 2024-08-25T21:09:39.049 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37108.log 2024-08-25T21:09:39.050 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85036.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85036.log.gz 2024-08-25T21:09:39.050 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105264.log 2024-08-25T21:09:39.051 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37108.log.gz 2024-08-25T21:09:39.051 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120154.log 2024-08-25T21:09:39.052 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105264.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105264.log.gz 2024-08-25T21:09:39.052 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27301.log 2024-08-25T21:09:39.052 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120154.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120154.log.gz 2024-08-25T21:09:39.053 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102741.log 2024-08-25T21:09:39.053 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27301.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27301.log.gz 2024-08-25T21:09:39.053 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134339.log 2024-08-25T21:09:39.054 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102741.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102741.log.gz 2024-08-25T21:09:39.054 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47075.log 2024-08-25T21:09:39.055 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134339.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134339.log.gz 2024-08-25T21:09:39.055 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96122.log 2024-08-25T21:09:39.056 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.47075.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47075.log.gz 2024-08-25T21:09:39.056 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137233.log 2024-08-25T21:09:39.056 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96122.log.gz 2024-08-25T21:09:39.056 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101491.log 2024-08-25T21:09:39.057 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137233.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137233.log.gz 2024-08-25T21:09:39.057 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40003.log 2024-08-25T21:09:39.058 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101491.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101491.log.gz 2024-08-25T21:09:39.058 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122511.log 2024-08-25T21:09:39.059 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40003.log.gz 2024-08-25T21:09:39.059 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78439.log 2024-08-25T21:09:39.059 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122511.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122511.log.gz 2024-08-25T21:09:39.060 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55949.log 2024-08-25T21:09:39.060 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78439.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.78439.log.gz 2024-08-25T21:09:39.060 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71222.log 2024-08-25T21:09:39.061 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55949.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55949.log.gz 2024-08-25T21:09:39.061 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88859.log 2024-08-25T21:09:39.062 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71222.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.71222.log.gz 2024-08-25T21:09:39.062 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.56037.log 2024-08-25T21:09:39.062 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88859.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80304.log 2024-08-25T21:09:39.063 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88859.log.gz 2024-08-25T21:09:39.063 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56037.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56037.log.gz 2024-08-25T21:09:39.063 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65749.log 2024-08-25T21:09:39.064 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80304.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80304.log.gz 2024-08-25T21:09:39.064 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111570.log 2024-08-25T21:09:39.065 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65749.log.gz 2024-08-25T21:09:39.065 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118515.log 2024-08-25T21:09:39.066 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111570.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111570.log.gz 2024-08-25T21:09:39.066 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61926.log 2024-08-25T21:09:39.066 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.118515.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118515.log.gz 2024-08-25T21:09:39.066 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.33437.log 2024-08-25T21:09:39.067 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61926.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61926.log.gz 2024-08-25T21:09:39.067 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98128.log 2024-08-25T21:09:39.068 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.33437.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.33437.log.gz 2024-08-25T21:09:39.068 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88445.log 2024-08-25T21:09:39.069 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98128.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98128.log.gz 2024-08-25T21:09:39.069 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83680.log 2024-08-25T21:09:39.069 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88445.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88445.log.gz 2024-08-25T21:09:39.069 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48968.log 2024-08-25T21:09:39.070 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83680.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83680.log.gz 2024-08-25T21:09:39.070 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98507.log 2024-08-25T21:09:39.071 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48968.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135071.log 2024-08-25T21:09:39.071 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48968.log.gz 2024-08-25T21:09:39.072 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98507.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98507.log.gz 2024-08-25T21:09:39.072 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141698.log 2024-08-25T21:09:39.073 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135071.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135071.log.gz 2024-08-25T21:09:39.073 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96620.log 2024-08-25T21:09:39.073 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.141698.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141698.log.gz 2024-08-25T21:09:39.073 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86358.log 2024-08-25T21:09:39.074 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96620.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96620.log.gz 2024-08-25T21:09:39.074 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72632.log 2024-08-25T21:09:39.075 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86358.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86358.log.gz 2024-08-25T21:09:39.075 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137320.log 2024-08-25T21:09:39.076 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72632.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72632.log.gz 2024-08-25T21:09:39.076 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109485.log 2024-08-25T21:09:39.076 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137320.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137320.log.gz 2024-08-25T21:09:39.076 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62936.log 2024-08-25T21:09:39.077 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109485.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109485.log.gz 2024-08-25T21:09:39.077 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51318.log 2024-08-25T21:09:39.078 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62936.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62936.log.gz 2024-08-25T21:09:39.078 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80612.log 2024-08-25T21:09:39.078 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51318.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51318.log.gz 2024-08-25T21:09:39.079 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52840.log 2024-08-25T21:09:39.079 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80612.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80612.log.gz 2024-08-25T21:09:39.079 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49218.log 2024-08-25T21:09:39.080 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52840.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52840.log.gz 2024-08-25T21:09:39.080 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130819.log 2024-08-25T21:09:39.081 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49218.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49218.log.gz 2024-08-25T21:09:39.081 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123120.log 2024-08-25T21:09:39.082 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130819.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130819.log.gz 2024-08-25T21:09:39.082 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77487.log 2024-08-25T21:09:39.082 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123120.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123120.log.gz 2024-08-25T21:09:39.082 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132624.log 2024-08-25T21:09:39.084 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77487.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77487.log.gz 2024-08-25T21:09:39.084 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134172.log 2024-08-25T21:09:39.084 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132624.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132624.log.gz 2024-08-25T21:09:39.084 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142315.log 2024-08-25T21:09:39.085 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134172.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134172.log.gz 2024-08-25T21:09:39.085 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79747.log 2024-08-25T21:09:39.086 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.142315.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142315.log.gz 2024-08-25T21:09:39.086 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133741.log 2024-08-25T21:09:39.086 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79747.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79747.log.gz 2024-08-25T21:09:39.087 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42860.log 2024-08-25T21:09:39.087 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133741.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133741.log.gz 2024-08-25T21:09:39.088 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121251.log 2024-08-25T21:09:39.088 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.42860.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42860.log.gz 2024-08-25T21:09:39.088 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100682.log 2024-08-25T21:09:39.089 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121251.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121251.log.gz 2024-08-25T21:09:39.089 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52538.log 2024-08-25T21:09:39.089 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100682.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100682.log.gz 2024-08-25T21:09:39.090 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96703.log 2024-08-25T21:09:39.090 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52538.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52538.log.gz 2024-08-25T21:09:39.090 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48654.log 2024-08-25T21:09:39.091 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96703.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96703.log.gz 2024-08-25T21:09:39.091 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134444.log 2024-08-25T21:09:39.092 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48654.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71306.log 2024-08-25T21:09:39.092 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48654.log.gz 2024-08-25T21:09:39.093 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134444.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134444.log.gz 2024-08-25T21:09:39.093 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133244.log 2024-08-25T21:09:39.093 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.71306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71306.log.gz 2024-08-25T21:09:39.094 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83159.log 2024-08-25T21:09:39.094 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133244.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133244.log.gz 2024-08-25T21:09:39.094 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.59713.log 2024-08-25T21:09:39.095 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.83159.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83159.log.gz 2024-08-25T21:09:39.095 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103232.log 2024-08-25T21:09:39.096 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.59713.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59713.log.gz 2024-08-25T21:09:39.096 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38828.log 2024-08-25T21:09:39.097 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103232.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103232.log.gz 2024-08-25T21:09:39.097 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38659.log 2024-08-25T21:09:39.098 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38828.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38828.log.gz 2024-08-25T21:09:39.098 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125772.log 2024-08-25T21:09:39.098 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38659.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38659.log.gz 2024-08-25T21:09:39.099 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125893.log 2024-08-25T21:09:39.100 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125772.log.gz 2024-08-25T21:09:39.100 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70994.log 2024-08-25T21:09:39.100 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125893.log.gz 2024-08-25T21:09:39.100 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28101.log 2024-08-25T21:09:39.101 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70994.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70994.log.gz 2024-08-25T21:09:39.101 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110630.log 2024-08-25T21:09:39.102 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28101.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28101.log.gz 2024-08-25T21:09:39.102 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87641.log 2024-08-25T21:09:39.102 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110630.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110630.log.gz 2024-08-25T21:09:39.103 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42913.log 2024-08-25T21:09:39.103 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87641.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87641.log.gz 2024-08-25T21:09:39.103 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84406.log 2024-08-25T21:09:39.104 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.42913.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42913.log.gz 2024-08-25T21:09:39.104 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36941.log 2024-08-25T21:09:39.105 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84406.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84406.log.gz 2024-08-25T21:09:39.105 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123913.log 2024-08-25T21:09:39.106 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36941.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36941.log.gz 2024-08-25T21:09:39.106 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49555.log 2024-08-25T21:09:39.107 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123913.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123913.log.gz 2024-08-25T21:09:39.107 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106127.log 2024-08-25T21:09:39.107 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49555.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49555.log.gz 2024-08-25T21:09:39.107 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44097.log 2024-08-25T21:09:39.108 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106127.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123663.log 2024-08-25T21:09:39.108 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106127.log.gz 2024-08-25T21:09:39.109 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44097.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44097.log.gz 2024-08-25T21:09:39.109 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63690.log 2024-08-25T21:09:39.110 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123663.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123663.log.gz 2024-08-25T21:09:39.110 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57188.log 2024-08-25T21:09:39.111 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63690.log.gz 2024-08-25T21:09:39.111 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91015.log 2024-08-25T21:09:39.111 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57188.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57188.log.gz 2024-08-25T21:09:39.111 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124984.log 2024-08-25T21:09:39.112 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91015.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91015.log.gz 2024-08-25T21:09:39.112 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26883.log 2024-08-25T21:09:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124984.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124984.log.gz 2024-08-25T21:09:39.113 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121503.log 2024-08-25T21:09:39.114 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26883.log.gz 2024-08-25T21:09:39.114 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73241.log 2024-08-25T21:09:39.115 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121503.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121503.log.gz 2024-08-25T21:09:39.115 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33532.log 2024-08-25T21:09:39.116 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73241.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73241.log.gz 2024-08-25T21:09:39.116 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117518.log 2024-08-25T21:09:39.116 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33532.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33532.log.gz 2024-08-25T21:09:39.117 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126362.log 2024-08-25T21:09:39.118 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117518.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117518.log.gz 2024-08-25T21:09:39.118 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27931.log 2024-08-25T21:09:39.119 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126362.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126362.log.gz 2024-08-25T21:09:39.119 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54962.log 2024-08-25T21:09:39.120 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27931.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27931.log.gz 2024-08-25T21:09:39.120 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118469.log 2024-08-25T21:09:39.120 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54962.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54962.log.gz 2024-08-25T21:09:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48507.log 2024-08-25T21:09:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118469.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118469.log.gz 2024-08-25T21:09:39.121 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100964.log 2024-08-25T21:09:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48507.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48507.log.gz 2024-08-25T21:09:39.122 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51824.log 2024-08-25T21:09:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100964.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100964.log.gz 2024-08-25T21:09:39.123 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104652.log 2024-08-25T21:09:39.124 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51824.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51824.log.gz 2024-08-25T21:09:39.124 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47711.log 2024-08-25T21:09:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104652.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104652.log.gz 2024-08-25T21:09:39.125 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63818.log 2024-08-25T21:09:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47711.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47711.log.gz 2024-08-25T21:09:39.126 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64026.log 2024-08-25T21:09:39.127 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63818.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63818.log.gz 2024-08-25T21:09:39.127 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28062.log 2024-08-25T21:09:39.128 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64026.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64026.log.gz 2024-08-25T21:09:39.128 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57058.log 2024-08-25T21:09:39.129 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28062.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28062.log.gz 2024-08-25T21:09:39.129 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137549.log 2024-08-25T21:09:39.129 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57058.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57058.log.gz 2024-08-25T21:09:39.130 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118422.log 2024-08-25T21:09:39.131 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137549.log.gz 2024-08-25T21:09:39.131 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112003.log 2024-08-25T21:09:39.132 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118422.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118422.log.gz 2024-08-25T21:09:39.132 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57752.log 2024-08-25T21:09:39.132 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112003.log.gz 2024-08-25T21:09:39.133 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137383.log 2024-08-25T21:09:39.133 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57752.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57752.log.gz 2024-08-25T21:09:39.134 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47125.log 2024-08-25T21:09:39.135 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137383.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137383.log.gz 2024-08-25T21:09:39.135 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68436.log 2024-08-25T21:09:39.136 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47125.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47125.log.gz 2024-08-25T21:09:39.136 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140718.log 2024-08-25T21:09:39.137 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68436.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68436.log.gz 2024-08-25T21:09:39.137 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131715.log 2024-08-25T21:09:39.138 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140718.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140718.log.gz 2024-08-25T21:09:39.138 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119255.log 2024-08-25T21:09:39.139 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131715.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131715.log.gz 2024-08-25T21:09:39.139 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129165.log 2024-08-25T21:09:39.139 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119255.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119255.log.gz 2024-08-25T21:09:39.140 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104503.log 2024-08-25T21:09:39.140 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.129165.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129165.log.gz 2024-08-25T21:09:39.140 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40844.log 2024-08-25T21:09:39.141 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.104503.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104503.log.gz 2024-08-25T21:09:39.141 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136062.log 2024-08-25T21:09:39.142 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40844.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40844.log.gz 2024-08-25T21:09:39.142 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118194.log 2024-08-25T21:09:39.143 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136062.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136062.log.gz 2024-08-25T21:09:39.143 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100357.log 2024-08-25T21:09:39.144 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118194.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118194.log.gz 2024-08-25T21:09:39.144 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86568.log 2024-08-25T21:09:39.145 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100357.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100357.log.gz 2024-08-25T21:09:39.145 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111806.log 2024-08-25T21:09:39.146 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86568.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86568.log.gz 2024-08-25T21:09:39.146 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71504.log 2024-08-25T21:09:39.147 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111806.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111806.log.gz 2024-08-25T21:09:39.147 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122092.log 2024-08-25T21:09:39.148 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71504.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71504.log.gz 2024-08-25T21:09:39.148 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85329.log 2024-08-25T21:09:39.149 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122092.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122092.log.gz 2024-08-25T21:09:39.149 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35141.log 2024-08-25T21:09:39.150 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85329.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85329.log.gz 2024-08-25T21:09:39.150 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78738.log 2024-08-25T21:09:39.150 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35141.log.gz 2024-08-25T21:09:39.151 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84951.log 2024-08-25T21:09:39.152 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78738.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78738.log.gz 2024-08-25T21:09:39.152 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40047.log 2024-08-25T21:09:39.152 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84951.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84951.log.gz 2024-08-25T21:09:39.152 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44666.log 2024-08-25T21:09:39.153 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40047.log.gz 2024-08-25T21:09:39.154 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52410.log 2024-08-25T21:09:39.154 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44666.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44666.log.gz 2024-08-25T21:09:39.155 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118495.log 2024-08-25T21:09:39.155 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52410.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52410.log.gz 2024-08-25T21:09:39.156 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67250.log 2024-08-25T21:09:39.156 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.118495.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118495.log.gz 2024-08-25T21:09:39.157 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64028.log 2024-08-25T21:09:39.157 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67250.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67250.log.gz 2024-08-25T21:09:39.157 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117671.log 2024-08-25T21:09:39.158 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64028.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64028.log.gz 2024-08-25T21:09:39.159 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82792.log 2024-08-25T21:09:39.159 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.117671.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117671.log.gz 2024-08-25T21:09:39.159 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46614.log 2024-08-25T21:09:39.160 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82792.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82792.log.gz 2024-08-25T21:09:39.160 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123710.log 2024-08-25T21:09:39.161 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46614.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46614.log.gz 2024-08-25T21:09:39.161 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87725.log 2024-08-25T21:09:39.162 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123710.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123710.log.gz 2024-08-25T21:09:39.162 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130757.log 2024-08-25T21:09:39.163 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87725.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87725.log.gz 2024-08-25T21:09:39.163 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62242.log 2024-08-25T21:09:39.164 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130757.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130757.log.gz 2024-08-25T21:09:39.164 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125385.log 2024-08-25T21:09:39.165 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62242.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62242.log.gz 2024-08-25T21:09:39.165 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99117.log 2024-08-25T21:09:39.165 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125385.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125385.log.gz 2024-08-25T21:09:39.166 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87135.log 2024-08-25T21:09:39.167 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99117.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99117.log.gz 2024-08-25T21:09:39.167 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116323.log 2024-08-25T21:09:39.167 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87135.log.gz 2024-08-25T21:09:39.168 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97630.log 2024-08-25T21:09:39.169 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.116323.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.116323.log.gz 2024-08-25T21:09:39.169 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106261.log 2024-08-25T21:09:39.169 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97630.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97630.log.gz 2024-08-25T21:09:39.170 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26522.log 2024-08-25T21:09:39.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106261.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106261.log.gz 2024-08-25T21:09:39.171 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103626.log 2024-08-25T21:09:39.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26522.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26522.log.gz 2024-08-25T21:09:39.172 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60815.log 2024-08-25T21:09:39.172 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103626.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103626.log.gz 2024-08-25T21:09:39.172 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43025.log 2024-08-25T21:09:39.173 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60815.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60815.log.gz 2024-08-25T21:09:39.173 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77973.log 2024-08-25T21:09:39.174 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43025.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43025.log.gz 2024-08-25T21:09:39.174 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88625.log 2024-08-25T21:09:39.175 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77973.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77973.log.gz 2024-08-25T21:09:39.175 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31463.log 2024-08-25T21:09:39.176 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88625.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88625.log.gz 2024-08-25T21:09:39.176 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125075.log 2024-08-25T21:09:39.177 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31463.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31463.log.gz 2024-08-25T21:09:39.177 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73667.log 2024-08-25T21:09:39.178 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125075.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125075.log.gz 2024-08-25T21:09:39.178 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91204.log 2024-08-25T21:09:39.179 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73667.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73667.log.gz 2024-08-25T21:09:39.179 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100924.log 2024-08-25T21:09:39.180 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91204.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91204.log.gz 2024-08-25T21:09:39.180 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79328.log 2024-08-25T21:09:39.181 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100924.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100924.log.gz 2024-08-25T21:09:39.181 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40633.log 2024-08-25T21:09:39.182 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79328.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79328.log.gz 2024-08-25T21:09:39.182 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68803.log 2024-08-25T21:09:39.183 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40633.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40633.log.gz 2024-08-25T21:09:39.183 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34095.log 2024-08-25T21:09:39.183 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68803.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68803.log.gz 2024-08-25T21:09:39.184 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109952.log 2024-08-25T21:09:39.184 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.34095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34095.log.gz 2024-08-25T21:09:39.185 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110412.log 2024-08-25T21:09:39.186 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109952.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109952.log.gz 2024-08-25T21:09:39.186 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114476.log 2024-08-25T21:09:39.187 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110412.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110412.log.gz 2024-08-25T21:09:39.187 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77573.log 2024-08-25T21:09:39.187 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114476.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114476.log.gz 2024-08-25T21:09:39.188 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.130227.log 2024-08-25T21:09:39.188 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77573.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77573.log.gz 2024-08-25T21:09:39.189 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54784.log 2024-08-25T21:09:39.189 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.130227.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.130227.log.gz 2024-08-25T21:09:39.189 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110160.log 2024-08-25T21:09:39.190 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54784.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54784.log.gz 2024-08-25T21:09:39.190 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134526.log 2024-08-25T21:09:39.191 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110160.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110160.log.gz 2024-08-25T21:09:39.191 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102101.log 2024-08-25T21:09:39.192 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134526.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134526.log.gz 2024-08-25T21:09:39.193 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83642.log 2024-08-25T21:09:39.193 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102101.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102101.log.gz 2024-08-25T21:09:39.193 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43697.log 2024-08-25T21:09:39.194 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83642.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83642.log.gz 2024-08-25T21:09:39.194 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38449.log 2024-08-25T21:09:39.195 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43697.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43697.log.gz 2024-08-25T21:09:39.195 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78674.log 2024-08-25T21:09:39.196 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38449.log.gz 2024-08-25T21:09:39.196 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122342.log 2024-08-25T21:09:39.197 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78674.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78674.log.gz 2024-08-25T21:09:39.197 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80251.log 2024-08-25T21:09:39.198 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122342.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122342.log.gz 2024-08-25T21:09:39.198 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55927.log 2024-08-25T21:09:39.199 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80251.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80251.log.gz 2024-08-25T21:09:39.199 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40594.log 2024-08-25T21:09:39.200 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55927.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55927.log.gz 2024-08-25T21:09:39.200 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83596.log 2024-08-25T21:09:39.201 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40594.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40594.log.gz 2024-08-25T21:09:39.201 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50352.log 2024-08-25T21:09:39.202 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.83596.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83596.log.gz 2024-08-25T21:09:39.202 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141543.log 2024-08-25T21:09:39.203 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50352.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50352.log.gz 2024-08-25T21:09:39.203 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86944.log 2024-08-25T21:09:39.204 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141543.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141543.log.gz 2024-08-25T21:09:39.204 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100253.log 2024-08-25T21:09:39.204 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86944.log.gz 2024-08-25T21:09:39.205 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38597.log 2024-08-25T21:09:39.205 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100253.log.gz 2024-08-25T21:09:39.205 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142449.log 2024-08-25T21:09:39.206 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38597.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38597.log.gz 2024-08-25T21:09:39.207 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57059.log 2024-08-25T21:09:39.207 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.142449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142449.log.gz 2024-08-25T21:09:39.208 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139141.log 2024-08-25T21:09:39.208 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57059.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57059.log.gz 2024-08-25T21:09:39.208 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43894.log 2024-08-25T21:09:39.209 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139141.log.gz 2024-08-25T21:09:39.209 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29042.log 2024-08-25T21:09:39.210 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43894.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43894.log.gz 2024-08-25T21:09:39.210 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140972.log 2024-08-25T21:09:39.211 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29042.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29042.log.gz 2024-08-25T21:09:39.211 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110863.log 2024-08-25T21:09:39.212 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.140972.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140972.log.gz 2024-08-25T21:09:39.212 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98970.log 2024-08-25T21:09:39.213 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110863.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110863.log.gz 2024-08-25T21:09:39.213 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135493.log 2024-08-25T21:09:39.214 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98970.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98970.log.gz 2024-08-25T21:09:39.214 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115254.log 2024-08-25T21:09:39.215 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135493.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135493.log.gz 2024-08-25T21:09:39.215 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108370.log 2024-08-25T21:09:39.216 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115254.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115254.log.gz 2024-08-25T21:09:39.216 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46903.log 2024-08-25T21:09:39.217 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108370.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108370.log.gz 2024-08-25T21:09:39.217 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97965.log 2024-08-25T21:09:39.218 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46903.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46903.log.gz 2024-08-25T21:09:39.218 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80204.log 2024-08-25T21:09:39.219 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97965.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97965.log.gz 2024-08-25T21:09:39.219 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129885.log 2024-08-25T21:09:39.219 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80204.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80204.log.gz 2024-08-25T21:09:39.220 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38050.log 2024-08-25T21:09:39.221 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129885.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129885.log.gz 2024-08-25T21:09:39.221 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33134.log 2024-08-25T21:09:39.222 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38050.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38050.log.gz 2024-08-25T21:09:39.222 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118864.log 2024-08-25T21:09:39.223 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33134.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33134.log.gz 2024-08-25T21:09:39.223 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28481.log 2024-08-25T21:09:39.223 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118864.log.gz 2024-08-25T21:09:39.224 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121967.log 2024-08-25T21:09:39.224 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28481.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28481.log.gz 2024-08-25T21:09:39.225 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33898.log 2024-08-25T21:09:39.226 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121967.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121967.log.gz 2024-08-25T21:09:39.226 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119328.log 2024-08-25T21:09:39.227 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33898.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33898.log.gz 2024-08-25T21:09:39.227 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141198.log 2024-08-25T21:09:39.228 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119328.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119328.log.gz 2024-08-25T21:09:39.228 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103996.log 2024-08-25T21:09:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141198.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141198.log.gz 2024-08-25T21:09:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.43178.log 2024-08-25T21:09:39.229 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.103996.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.103996.log.gz 2024-08-25T21:09:39.230 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110652.log 2024-08-25T21:09:39.230 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.43178.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.43178.log.gz 2024-08-25T21:09:39.230 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89657.log 2024-08-25T21:09:39.231 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110652.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110652.log.gz 2024-08-25T21:09:39.232 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84473.log 2024-08-25T21:09:39.232 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89657.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89657.log.gz 2024-08-25T21:09:39.232 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132816.log 2024-08-25T21:09:39.233 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84473.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84473.log.gz 2024-08-25T21:09:39.233 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93984.log 2024-08-25T21:09:39.234 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132816.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132816.log.gz 2024-08-25T21:09:39.234 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27261.log 2024-08-25T21:09:39.235 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.93984.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93984.log.gz 2024-08-25T21:09:39.235 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113215.log 2024-08-25T21:09:39.236 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27261.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27261.log.gz 2024-08-25T21:09:39.236 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120851.log 2024-08-25T21:09:39.237 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113215.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113215.log.gz 2024-08-25T21:09:39.237 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40341.log 2024-08-25T21:09:39.238 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120851.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120851.log.gz 2024-08-25T21:09:39.238 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89469.log 2024-08-25T21:09:39.239 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40341.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40341.log.gz 2024-08-25T21:09:39.239 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38387.log 2024-08-25T21:09:39.240 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89469.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89469.log.gz 2024-08-25T21:09:39.240 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137092.log 2024-08-25T21:09:39.241 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38387.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38387.log.gz 2024-08-25T21:09:39.241 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111052.log 2024-08-25T21:09:39.241 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137092.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137092.log.gz 2024-08-25T21:09:39.242 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81880.log 2024-08-25T21:09:39.243 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111052.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111052.log.gz 2024-08-25T21:09:39.243 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114114.log 2024-08-25T21:09:39.244 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.81880.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81880.log.gz 2024-08-25T21:09:39.244 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55840.log 2024-08-25T21:09:39.245 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114114.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114114.log.gz 2024-08-25T21:09:39.245 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99956.log 2024-08-25T21:09:39.246 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55840.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55840.log.gz 2024-08-25T21:09:39.246 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60149.log 2024-08-25T21:09:39.247 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99956.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99956.log.gz 2024-08-25T21:09:39.247 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66210.log 2024-08-25T21:09:39.247 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60149.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60149.log.gz 2024-08-25T21:09:39.247 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89466.log 2024-08-25T21:09:39.248 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66210.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66210.log.gz 2024-08-25T21:09:39.248 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35919.log 2024-08-25T21:09:39.249 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89466.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89466.log.gz 2024-08-25T21:09:39.249 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113484.log 2024-08-25T21:09:39.250 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.35919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35919.log.gz 2024-08-25T21:09:39.250 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45852.log 2024-08-25T21:09:39.251 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113484.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113484.log.gz 2024-08-25T21:09:39.257 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58654.log 2024-08-25T21:09:39.257 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45852.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45852.log.gz 2024-08-25T21:09:39.257 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135577.log 2024-08-25T21:09:39.257 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58654.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58654.log.gz 2024-08-25T21:09:39.257 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78562.log 2024-08-25T21:09:39.257 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135577.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135577.log.gz 2024-08-25T21:09:39.258 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76061.log 2024-08-25T21:09:39.258 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78562.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78562.log.gz 2024-08-25T21:09:39.258 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64698.log 2024-08-25T21:09:39.258 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76061.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76061.log.gz 2024-08-25T21:09:39.258 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132231.log 2024-08-25T21:09:39.258 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64698.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64698.log.gz 2024-08-25T21:09:39.258 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27134.log 2024-08-25T21:09:39.260 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132231.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132231.log.gz 2024-08-25T21:09:39.260 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75092.log 2024-08-25T21:09:39.260 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27134.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27134.log.gz 2024-08-25T21:09:39.260 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96283.log 2024-08-25T21:09:39.260 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75092.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75092.log.gz 2024-08-25T21:09:39.260 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53378.log 2024-08-25T21:09:39.261 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96283.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96283.log.gz 2024-08-25T21:09:39.261 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57181.log 2024-08-25T21:09:39.262 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53378.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53378.log.gz 2024-08-25T21:09:39.262 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29950.log 2024-08-25T21:09:39.262 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57181.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57181.log.gz 2024-08-25T21:09:39.263 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53966.log 2024-08-25T21:09:39.263 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29950.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29950.log.gz 2024-08-25T21:09:39.264 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27871.log 2024-08-25T21:09:39.264 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53966.log.gz 2024-08-25T21:09:39.264 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101203.log 2024-08-25T21:09:39.265 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27871.log.gz 2024-08-25T21:09:39.265 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120935.log 2024-08-25T21:09:39.266 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101203.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101203.log.gz 2024-08-25T21:09:39.266 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141859.log 2024-08-25T21:09:39.267 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120935.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120935.log.gz 2024-08-25T21:09:39.267 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96765.log 2024-08-25T21:09:39.268 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141859.log.gz 2024-08-25T21:09:39.268 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101806.log 2024-08-25T21:09:39.269 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96765.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96765.log.gz 2024-08-25T21:09:39.269 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66001.log 2024-08-25T21:09:39.270 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101806.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101806.log.gz 2024-08-25T21:09:39.270 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26468.log 2024-08-25T21:09:39.271 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66001.log.gz 2024-08-25T21:09:39.271 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141604.log 2024-08-25T21:09:39.271 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26468.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26468.log.gz 2024-08-25T21:09:39.272 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44151.log 2024-08-25T21:09:39.273 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141604.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141604.log.gz 2024-08-25T21:09:39.273 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61486.log 2024-08-25T21:09:39.274 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44151.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44151.log.gz 2024-08-25T21:09:39.274 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110014.log 2024-08-25T21:09:39.275 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61486.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61486.log.gz 2024-08-25T21:09:39.275 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58680.log 2024-08-25T21:09:39.275 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110014.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110014.log.gz 2024-08-25T21:09:39.276 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26338.log 2024-08-25T21:09:39.276 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58680.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58680.log.gz 2024-08-25T21:09:39.277 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57730.log 2024-08-25T21:09:39.278 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26338.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26338.log.gz 2024-08-25T21:09:39.278 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25462.log 2024-08-25T21:09:39.278 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57730.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57730.log.gz 2024-08-25T21:09:39.279 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106220.log 2024-08-25T21:09:39.279 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25462.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25462.log.gz 2024-08-25T21:09:39.279 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.142133.log 2024-08-25T21:09:39.280 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.106220.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106220.log.gz 2024-08-25T21:09:39.280 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123162.log 2024-08-25T21:09:39.281 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.142133.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.142133.log.gz 2024-08-25T21:09:39.282 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133951.log 2024-08-25T21:09:39.282 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123162.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123162.log.gz 2024-08-25T21:09:39.282 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65454.log 2024-08-25T21:09:39.283 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133951.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133951.log.gz 2024-08-25T21:09:39.283 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125218.log 2024-08-25T21:09:39.284 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65454.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65454.log.gz 2024-08-25T21:09:39.284 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73010.log 2024-08-25T21:09:39.285 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125218.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125218.log.gz 2024-08-25T21:09:39.285 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34853.log 2024-08-25T21:09:39.286 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73010.log.gz 2024-08-25T21:09:39.286 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69514.log 2024-08-25T21:09:39.287 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34853.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34853.log.gz 2024-08-25T21:09:39.287 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138384.log 2024-08-25T21:09:39.288 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.69514.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52662.log 2024-08-25T21:09:39.288 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69514.log.gz 2024-08-25T21:09:39.289 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138384.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138384.log.gz 2024-08-25T21:09:39.289 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45672.log 2024-08-25T21:09:39.290 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52662.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52662.log.gz 2024-08-25T21:09:39.290 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75974.log 2024-08-25T21:09:39.291 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45672.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45672.log.gz 2024-08-25T21:09:39.291 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119912.log 2024-08-25T21:09:39.292 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75974.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75974.log.gz 2024-08-25T21:09:39.292 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53126.log 2024-08-25T21:09:39.292 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119912.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119912.log.gz 2024-08-25T21:09:39.292 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117697.log 2024-08-25T21:09:39.293 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53126.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53126.log.gz 2024-08-25T21:09:39.293 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45038.log 2024-08-25T21:09:39.294 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.117697.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117697.log.gz 2024-08-25T21:09:39.294 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122277.log 2024-08-25T21:09:39.295 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.45038.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45038.log.gz 2024-08-25T21:09:39.295 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130154.log 2024-08-25T21:09:39.296 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122277.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122277.log.gz 2024-08-25T21:09:39.296 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62893.log 2024-08-25T21:09:39.297 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.130154.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130154.log.gz 2024-08-25T21:09:39.297 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127142.log 2024-08-25T21:09:39.297 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62893.log.gz 2024-08-25T21:09:39.298 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29235.log 2024-08-25T21:09:39.299 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127142.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127142.log.gz 2024-08-25T21:09:39.299 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50122.log 2024-08-25T21:09:39.300 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29235.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29235.log.gz 2024-08-25T21:09:39.300 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103711.log 2024-08-25T21:09:39.301 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50122.log.gz 2024-08-25T21:09:39.301 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53796.log 2024-08-25T21:09:39.301 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103711.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103711.log.gz 2024-08-25T21:09:39.302 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140372.log 2024-08-25T21:09:39.302 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53796.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53796.log.gz 2024-08-25T21:09:39.303 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72379.log 2024-08-25T21:09:39.304 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.140372.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140372.log.gz 2024-08-25T21:09:39.304 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106656.log 2024-08-25T21:09:39.304 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72379.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72379.log.gz 2024-08-25T21:09:39.305 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.60385.log 2024-08-25T21:09:39.305 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.106656.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106656.log.gz 2024-08-25T21:09:39.305 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132504.log 2024-08-25T21:09:39.306 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.60385.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.60385.log.gz 2024-08-25T21:09:39.306 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130623.log 2024-08-25T21:09:39.307 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132504.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132504.log.gz 2024-08-25T21:09:39.307 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101028.log 2024-08-25T21:09:39.308 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130623.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130623.log.gz 2024-08-25T21:09:39.308 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.95824.log 2024-08-25T21:09:39.309 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101028.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101028.log.gz 2024-08-25T21:09:39.309 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57472.log 2024-08-25T21:09:39.310 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.95824.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95824.log.gz 2024-08-25T21:09:39.310 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77760.log 2024-08-25T21:09:39.310 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.57472.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57472.log.gz 2024-08-25T21:09:39.310 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123940.log 2024-08-25T21:09:39.311 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77760.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.mirror.77760.log.gz /var/log/ceph/cluster1-client.mirror.98384.log 2024-08-25T21:09:39.311 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:39.312 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123940.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.mirror.123940.log.gz /var/log/ceph/cluster2-client.admin.117452.log 2024-08-25T21:09:39.312 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:39.312 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98384.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98384.log.gz 2024-08-25T21:09:39.312 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135391.log 2024-08-25T21:09:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117452.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117452.log.gz 2024-08-25T21:09:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61380.log 2024-08-25T21:09:39.314 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135391.log.gz 2024-08-25T21:09:39.314 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86800.log 2024-08-25T21:09:39.315 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61380.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61380.log.gz 2024-08-25T21:09:39.315 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70444.log 2024-08-25T21:09:39.315 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86800.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86800.log.gz 2024-08-25T21:09:39.315 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106911.log 2024-08-25T21:09:39.316 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70444.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70444.log.gz 2024-08-25T21:09:39.316 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42546.log 2024-08-25T21:09:39.317 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106911.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106911.log.gz 2024-08-25T21:09:39.317 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59475.log 2024-08-25T21:09:39.318 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42546.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42546.log.gz 2024-08-25T21:09:39.318 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111426.log 2024-08-25T21:09:39.318 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59475.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59475.log.gz 2024-08-25T21:09:39.318 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50227.log 2024-08-25T21:09:39.319 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111426.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111426.log.gz 2024-08-25T21:09:39.319 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133279.log 2024-08-25T21:09:39.320 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50227.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50227.log.gz 2024-08-25T21:09:39.320 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22182.log 2024-08-25T21:09:39.321 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133279.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133279.log.gz 2024-08-25T21:09:39.321 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85296.log 2024-08-25T21:09:39.321 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22182.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster2-client.admin.22182.log.gz -- /var/log/ceph/cluster1-client.admin.141946.log 2024-08-25T21:09:39.321 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:39.322 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85296.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85296.log.gz 2024-08-25T21:09:39.322 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79329.log 2024-08-25T21:09:39.323 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141946.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141946.log.gz 2024-08-25T21:09:39.323 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35028.log 2024-08-25T21:09:39.324 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79329.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79329.log.gz 2024-08-25T21:09:39.324 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109297.log 2024-08-25T21:09:39.324 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.35028.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35028.log.gz 2024-08-25T21:09:39.324 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109734.log 2024-08-25T21:09:39.325 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109297.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109297.log.gz 2024-08-25T21:09:39.325 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95923.log 2024-08-25T21:09:39.326 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109734.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.109734.log.gz -5 2024-08-25T21:09:39.326 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster2-client.admin.105565.log 2024-08-25T21:09:39.327 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95923.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61298.log 2024-08-25T21:09:39.327 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95923.log.gz 2024-08-25T21:09:39.327 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105565.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105565.log.gz 2024-08-25T21:09:39.327 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71020.log 2024-08-25T21:09:39.328 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61298.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61298.log.gz 2024-08-25T21:09:39.328 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123206.log 2024-08-25T21:09:39.329 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71020.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51202.log 2024-08-25T21:09:39.329 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71020.log.gz 2024-08-25T21:09:39.330 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123206.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123206.log.gz 2024-08-25T21:09:39.330 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57473.log 2024-08-25T21:09:39.330 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51202.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51202.log.gz 2024-08-25T21:09:39.330 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94414.log 2024-08-25T21:09:39.331 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.57473.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57473.log.gz 2024-08-25T21:09:39.331 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129074.log 2024-08-25T21:09:39.332 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94414.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94414.log.gz 2024-08-25T21:09:39.332 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121712.log 2024-08-25T21:09:39.333 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.129074.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129074.log.gz 2024-08-25T21:09:39.333 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45746.log 2024-08-25T21:09:39.334 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121712.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121712.log.gz 2024-08-25T21:09:39.334 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46292.log 2024-08-25T21:09:39.334 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45746.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.admin.45746.log.gz 2024-08-25T21:09:39.334 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30303.log 2024-08-25T21:09:39.335 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46292.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46292.log.gz 2024-08-25T21:09:39.335 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96449.log 2024-08-25T21:09:39.336 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30303.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30303.log.gz 2024-08-25T21:09:39.336 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44802.log 2024-08-25T21:09:39.336 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96449.log.gz 2024-08-25T21:09:39.336 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88269.log 2024-08-25T21:09:39.337 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44802.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44802.log.gz -5 2024-08-25T21:09:39.337 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.58949.log 2024-08-25T21:09:39.338 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88269.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88269.log.gz 2024-08-25T21:09:39.338 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76521.log 2024-08-25T21:09:39.339 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.58949.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58949.log.gz 2024-08-25T21:09:39.339 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37065.log 2024-08-25T21:09:39.340 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76521.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76521.log.gz 2024-08-25T21:09:39.340 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27032.log 2024-08-25T21:09:39.340 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37065.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.37065.log.gz -5 2024-08-25T21:09:39.340 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster2-client.admin.45405.log 2024-08-25T21:09:39.341 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27032.log.gz 2024-08-25T21:09:39.341 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89570.log 2024-08-25T21:09:39.342 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45405.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45405.log.gz 2024-08-25T21:09:39.342 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74066.log 2024-08-25T21:09:39.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89570.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74714.log 2024-08-25T21:09:39.343 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89570.log.gz 2024-08-25T21:09:39.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74066.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.74066.log.gz --verbose 2024-08-25T21:09:39.343 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster2-client.admin.81236.log 2024-08-25T21:09:39.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74714.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74714.log.gz 2024-08-25T21:09:39.344 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124146.log 2024-08-25T21:09:39.345 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81236.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82202.log 2024-08-25T21:09:39.345 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81236.log.gz 2024-08-25T21:09:39.346 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124146.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124146.log.gz 2024-08-25T21:09:39.346 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108415.log 2024-08-25T21:09:39.346 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82202.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82202.log.gz 2024-08-25T21:09:39.347 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.139580.log 2024-08-25T21:09:39.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108415.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108415.log.gz 2024-08-25T21:09:39.347 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61599.log 2024-08-25T21:09:39.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.139580.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.139580.log.gz 2024-08-25T21:09:39.348 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102438.log 2024-08-25T21:09:39.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61599.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61599.log.gz 2024-08-25T21:09:39.349 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112122.log 2024-08-25T21:09:39.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102438.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102438.log.gz 2024-08-25T21:09:39.350 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88272.log 2024-08-25T21:09:39.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112122.log.gz 2024-08-25T21:09:39.351 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51949.log 2024-08-25T21:09:39.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88272.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88272.log.gz 2024-08-25T21:09:39.352 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29970.log 2024-08-25T21:09:39.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51949.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51949.log.gz 2024-08-25T21:09:39.352 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89509.log 2024-08-25T21:09:39.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29970.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29970.log.gz 2024-08-25T21:09:39.353 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63733.log 2024-08-25T21:09:39.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89509.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89509.log.gz 2024-08-25T21:09:39.353 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133148.log 2024-08-25T21:09:39.354 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63733.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63733.log.gz 2024-08-25T21:09:39.354 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40173.log 2024-08-25T21:09:39.355 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133148.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133148.log.gz -5 2024-08-25T21:09:39.355 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.66230.log 2024-08-25T21:09:39.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40173.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40173.log.gz 2024-08-25T21:09:39.356 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74285.log 2024-08-25T21:09:39.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66230.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66230.log.gz 2024-08-25T21:09:39.357 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121353.log 2024-08-25T21:09:39.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74285.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74285.log.gz 2024-08-25T21:09:39.357 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36897.log 2024-08-25T21:09:39.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121353.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121353.log.gz 2024-08-25T21:09:39.358 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53546.log 2024-08-25T21:09:39.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36897.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36897.log.gz 2024-08-25T21:09:39.359 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27849.log 2024-08-25T21:09:39.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53546.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.admin.70619.log -- replaced with /var/log/ceph/cluster1-client.mirror.53546.log.gz 2024-08-25T21:09:39.360 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:39.360 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27849.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27849.log.gz 2024-08-25T21:09:39.360 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141885.log 2024-08-25T21:09:39.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70619.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.70619.log.gz -5 2024-08-25T21:09:39.361 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.28859.log 2024-08-25T21:09:39.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141885.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141885.log.gz 2024-08-25T21:09:39.362 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30560.log 2024-08-25T21:09:39.363 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28859.log.gz 2024-08-25T21:09:39.363 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88417.log 2024-08-25T21:09:39.363 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30560.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30560.log.gz 2024-08-25T21:09:39.363 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118583.log 2024-08-25T21:09:39.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88417.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88417.log.gz 2024-08-25T21:09:39.364 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141926.log 2024-08-25T21:09:39.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118583.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118583.log.gz 2024-08-25T21:09:39.365 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25272.log 2024-08-25T21:09:39.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.141926.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141926.log.gz 2024-08-25T21:09:39.366 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84408.log 2024-08-25T21:09:39.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25272.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25272.log.gz 2024-08-25T21:09:39.366 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28942.log 2024-08-25T21:09:39.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84408.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.84408.log.gz --verbose 2024-08-25T21:09:39.367 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.24790.log 2024-08-25T21:09:39.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28942.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28942.log.gz 2024-08-25T21:09:39.368 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42914.log 2024-08-25T21:09:39.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24790.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24790.log.gz 2024-08-25T21:09:39.369 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82404.log 2024-08-25T21:09:39.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.42914.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42914.log.gz 2024-08-25T21:09:39.370 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64827.log 2024-08-25T21:09:39.370 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82404.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.82404.log.gz --verbose 2024-08-25T21:09:39.370 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.admin.55731.log 2024-08-25T21:09:39.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64827.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64827.log.gz 2024-08-25T21:09:39.371 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126859.log 2024-08-25T21:09:39.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55731.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55731.log.gz 2024-08-25T21:09:39.372 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99576.log 2024-08-25T21:09:39.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126859.log.gz 2024-08-25T21:09:39.373 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107739.log 2024-08-25T21:09:39.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99576.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.99576.log.gz -5 2024-08-25T21:09:39.373 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.108352.log 2024-08-25T21:09:39.374 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107739.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107739.log.gz 2024-08-25T21:09:39.374 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104885.log 2024-08-25T21:09:39.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108352.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108352.log.gz 2024-08-25T21:09:39.376 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119718.log 2024-08-25T21:09:39.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104885.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104885.log.gz 2024-08-25T21:09:39.376 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82941.log 2024-08-25T21:09:39.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119718.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119718.log.gz 2024-08-25T21:09:39.377 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108290.log 2024-08-25T21:09:39.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82941.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82941.log.gz 2024-08-25T21:09:39.377 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63228.log 2024-08-25T21:09:39.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108290.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109926.log 2024-08-25T21:09:39.378 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108290.log.gz 2024-08-25T21:09:39.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63228.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63228.log.gz 2024-08-25T21:09:39.379 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42688.log 2024-08-25T21:09:39.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109926.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.109926.log.gz -- 2024-08-25T21:09:39.380 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.admin.42757.log 2024-08-25T21:09:39.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.42688.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42688.log.gz 2024-08-25T21:09:39.380 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132311.log 2024-08-25T21:09:39.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.42757.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42757.log.gz 2024-08-25T21:09:39.381 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72485.log 2024-08-25T21:09:39.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132311.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132311.log.gz 2024-08-25T21:09:39.382 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89195.log 2024-08-25T21:09:39.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72485.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72485.log.gz 2024-08-25T21:09:39.383 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98795.log 2024-08-25T21:09:39.383 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89195.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89195.log.gz 2024-08-25T21:09:39.383 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111430.log 2024-08-25T21:09:39.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98795.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98795.log.gz 2024-08-25T21:09:39.384 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124986.log 2024-08-25T21:09:39.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111430.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111430.log.gz 2024-08-25T21:09:39.385 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.33802.log 2024-08-25T21:09:39.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124986.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124986.log.gz 2024-08-25T21:09:39.386 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60320.log 2024-08-25T21:09:39.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.33802.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.33802.log.gz 2024-08-25T21:09:39.387 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40045.log 2024-08-25T21:09:39.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60320.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60320.log.gzgzip 2024-08-25T21:09:39.387 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.82176.log 2024-08-25T21:09:39.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40045.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.mirror.40045.log.gz /var/log/ceph/cluster1-client.mirror.77427.log 2024-08-25T21:09:39.388 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:39.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82176.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82176.log.gz 2024-08-25T21:09:39.389 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51699.log 2024-08-25T21:09:39.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77427.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77427.log.gz 2024-08-25T21:09:39.389 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126360.log 2024-08-25T21:09:39.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51699.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51699.log.gz 2024-08-25T21:09:39.390 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86065.log 2024-08-25T21:09:39.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126360.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126360.log.gz 2024-08-25T21:09:39.391 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98386.log 2024-08-25T21:09:39.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86065.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86065.log.gz 2024-08-25T21:09:39.392 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39290.log 2024-08-25T21:09:39.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98386.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98386.log.gz 2024-08-25T21:09:39.392 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126447.log 2024-08-25T21:09:39.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39290.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39290.log.gz 2024-08-25T21:09:39.393 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48196.log 2024-08-25T21:09:39.394 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.126447.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126447.log.gz 2024-08-25T21:09:39.394 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104505.log 2024-08-25T21:09:39.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48196.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48196.log.gz 2024-08-25T21:09:39.395 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117986.log 2024-08-25T21:09:39.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.104505.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104505.log.gz 2024-08-25T21:09:39.395 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53168.log 2024-08-25T21:09:39.396 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.117986.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.admin.117986.log.gz -- 2024-08-25T21:09:39.396 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.admin.59163.log 2024-08-25T21:09:39.397 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53168.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53168.log.gz 2024-08-25T21:09:39.397 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59473.log 2024-08-25T21:09:39.398 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.59163.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59163.log.gz 2024-08-25T21:09:39.398 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28626.log 2024-08-25T21:09:39.398 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59473.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59473.log.gz 2024-08-25T21:09:39.399 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36119.log 2024-08-25T21:09:39.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28626.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.28626.log.gz -5 2024-08-25T21:09:39.399 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.46883.log 2024-08-25T21:09:39.400 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36119.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36119.log.gz 2024-08-25T21:09:39.400 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93251.log 2024-08-25T21:09:39.400 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.46883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46883.log.gz 2024-08-25T21:09:39.401 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128928.log 2024-08-25T21:09:39.401 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93251.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93251.log.gz 2024-08-25T21:09:39.401 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69874.log 2024-08-25T21:09:39.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128928.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128928.log.gz 2024-08-25T21:09:39.402 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126518.log 2024-08-25T21:09:39.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.69874.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119864.log 2024-08-25T21:09:39.403 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69874.log.gz 2024-08-25T21:09:39.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126518.log.gz 2024-08-25T21:09:39.404 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28561.log 2024-08-25T21:09:39.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119864.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119864.log.gz -5 2024-08-25T21:09:39.405 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.133003.log 2024-08-25T21:09:39.405 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28561.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28561.log.gz 2024-08-25T21:09:39.405 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25999.log 2024-08-25T21:09:39.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133003.log.gz 2024-08-25T21:09:39.406 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32159.log 2024-08-25T21:09:39.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25999.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42483.log 2024-08-25T21:09:39.407 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25999.log.gz 2024-08-25T21:09:39.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32159.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32159.log.gz 2024-08-25T21:09:39.408 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63708.log 2024-08-25T21:09:39.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42483.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42483.log.gz 2024-08-25T21:09:39.409 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68963.log 2024-08-25T21:09:39.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63708.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.63708.log.gz --verbose 2024-08-25T21:09:39.409 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.88688.log 2024-08-25T21:09:39.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68963.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68963.log.gz 2024-08-25T21:09:39.410 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72903.log 2024-08-25T21:09:39.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88688.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88688.log.gz 2024-08-25T21:09:39.411 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47739.log 2024-08-25T21:09:39.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72903.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72903.log.gz 2024-08-25T21:09:39.412 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127076.log 2024-08-25T21:09:39.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47739.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47739.log.gz 2024-08-25T21:09:39.413 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141657.log 2024-08-25T21:09:39.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127076.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127076.log.gz 2024-08-25T21:09:39.414 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86066.log 2024-08-25T21:09:39.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141657.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141657.log.gz 2024-08-25T21:09:39.415 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134550.log 2024-08-25T21:09:39.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86066.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86066.log.gz 2024-08-25T21:09:39.415 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119105.log 2024-08-25T21:09:39.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134550.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.134550.log.gz -5 2024-08-25T21:09:39.416 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.49786.log 2024-08-25T21:09:39.416 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119105.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119105.log.gz 2024-08-25T21:09:39.416 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23775.log 2024-08-25T21:09:39.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49786.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49786.log.gz 2024-08-25T21:09:39.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85476.log 2024-08-25T21:09:39.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23775.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23775.log.gz 2024-08-25T21:09:39.418 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25000.log 2024-08-25T21:09:39.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85476.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85476.log.gz 2024-08-25T21:09:39.419 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51446.log 2024-08-25T21:09:39.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25000.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25000.log.gz 2024-08-25T21:09:39.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55311.log 2024-08-25T21:09:39.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51446.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51446.log.gz 2024-08-25T21:09:39.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131368.log 2024-08-25T21:09:39.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55311.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55311.log.gzgzip 2024-08-25T21:09:39.421 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101781.log 2024-08-25T21:09:39.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131368.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131368.log.gz 2024-08-25T21:09:39.422 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113001.log 2024-08-25T21:09:39.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101781.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101781.log.gz 2024-08-25T21:09:39.423 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32384.log 2024-08-25T21:09:39.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113001.log.gz 2024-08-25T21:09:39.423 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111805.log 2024-08-25T21:09:39.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32384.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32384.log.gz 2024-08-25T21:09:39.424 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68483.log 2024-08-25T21:09:39.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111805.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111805.log.gz 2024-08-25T21:09:39.425 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82678.log 2024-08-25T21:09:39.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68483.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68483.log.gz 2024-08-25T21:09:39.426 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103340.log 2024-08-25T21:09:39.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82678.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82678.log.gz 2024-08-25T21:09:39.427 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43652.log 2024-08-25T21:09:39.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103340.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103340.log.gz 2024-08-25T21:09:39.427 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37502.log 2024-08-25T21:09:39.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43652.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43652.log.gz 2024-08-25T21:09:39.428 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39333.log 2024-08-25T21:09:39.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37502.log.gz 2024-08-25T21:09:39.429 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35725.log 2024-08-25T21:09:39.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39333.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75642.log 2024-08-25T21:09:39.429 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39333.log.gz 2024-08-25T21:09:39.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35725.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35725.log.gz 2024-08-25T21:09:39.430 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38300.log 2024-08-25T21:09:39.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75642.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75642.log.gzgzip 2024-08-25T21:09:39.431 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.140592.log 2024-08-25T21:09:39.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38300.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38300.log.gz 2024-08-25T21:09:39.432 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91690.log 2024-08-25T21:09:39.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140592.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140592.log.gz 2024-08-25T21:09:39.433 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35055.log 2024-08-25T21:09:39.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91690.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster2-client.admin.128999.log 2024-08-25T21:09:39.433 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.91690.log.gz 2024-08-25T21:09:39.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.35055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35055.log.gz 2024-08-25T21:09:39.434 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25062.log 2024-08-25T21:09:39.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128999.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128999.log.gz 2024-08-25T21:09:39.435 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78201.log 2024-08-25T21:09:39.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25062.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126227.log 2024-08-25T21:09:39.436 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25062.log.gz 2024-08-25T21:09:39.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78201.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78201.log.gz 2024-08-25T21:09:39.436 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127385.log 2024-08-25T21:09:39.437 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126227.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126227.log.gz 2024-08-25T21:09:39.437 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29883.log 2024-08-25T21:09:39.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127385.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75975.log 2024-08-25T21:09:39.438 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127385.log.gz 2024-08-25T21:09:39.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29883.log.gz 2024-08-25T21:09:39.439 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56652.log 2024-08-25T21:09:39.440 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75975.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75975.log.gz 2024-08-25T21:09:39.440 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29278.log 2024-08-25T21:09:39.440 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56652.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56652.log.gz 2024-08-25T21:09:39.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61634.log 2024-08-25T21:09:39.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29278.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29278.log.gz 2024-08-25T21:09:39.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71458.log 2024-08-25T21:09:39.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61634.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61634.log.gz 2024-08-25T21:09:39.442 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50206.log 2024-08-25T21:09:39.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.71458.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71458.log.gz 2024-08-25T21:09:39.443 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63648.log 2024-08-25T21:09:39.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50206.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50206.log.gz 2024-08-25T21:09:39.444 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137968.log 2024-08-25T21:09:39.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63648.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63648.log.gz 2024-08-25T21:09:39.445 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60878.log 2024-08-25T21:09:39.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.137968.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137968.log.gz 2024-08-25T21:09:39.445 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100566.log 2024-08-25T21:09:39.446 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60878.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60878.log.gz 2024-08-25T21:09:39.446 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120706.log 2024-08-25T21:09:39.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100566.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100566.log.gz 2024-08-25T21:09:39.447 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.24022.log 2024-08-25T21:09:39.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120706.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120706.log.gz 2024-08-25T21:09:39.448 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75420.log 2024-08-25T21:09:39.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.24022.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.24022.log.gz 2024-08-25T21:09:39.449 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108775.log 2024-08-25T21:09:39.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75420.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75420.log.gz 2024-08-25T21:09:39.449 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99707.log 2024-08-25T21:09:39.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108775.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108775.log.gz 2024-08-25T21:09:39.450 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100314.log 2024-08-25T21:09:39.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99707.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99707.log.gz 2024-08-25T21:09:39.451 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55509.log 2024-08-25T21:09:39.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100314.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100314.log.gz 2024-08-25T21:09:39.452 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81668.log 2024-08-25T21:09:39.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55509.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55509.log.gz 2024-08-25T21:09:39.452 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37441.log 2024-08-25T21:09:39.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81668.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81668.log.gz 2024-08-25T21:09:39.454 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85937.log 2024-08-25T21:09:39.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37441.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37441.log.gz 2024-08-25T21:09:39.454 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77133.log 2024-08-25T21:09:39.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85937.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85937.log.gz 2024-08-25T21:09:39.454 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125186.log 2024-08-25T21:09:39.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77133.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77133.log.gz 2024-08-25T21:09:39.461 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63782.log 2024-08-25T21:09:39.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125186.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125186.log.gz 2024-08-25T21:09:39.461 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135434.log 2024-08-25T21:09:39.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63782.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63782.log.gz 2024-08-25T21:09:39.462 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66168.log 2024-08-25T21:09:39.462 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135434.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135434.log.gz -5 2024-08-25T21:09:39.462 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.132879.log 2024-08-25T21:09:39.462 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66168.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66168.log.gz 2024-08-25T21:09:39.462 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27973.log 2024-08-25T21:09:39.462 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132879.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132879.log.gz 2024-08-25T21:09:39.462 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105671.log 2024-08-25T21:09:39.462 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27973.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27973.log.gz 2024-08-25T21:09:39.462 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.139749.log 2024-08-25T21:09:39.462 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.105671.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.105671.log.gz 2024-08-25T21:09:39.462 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94574.log 2024-08-25T21:09:39.463 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.139749.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.139749.log.gz 2024-08-25T21:09:39.463 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120894.log 2024-08-25T21:09:39.464 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94574.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40804.log 2024-08-25T21:09:39.464 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94574.log.gz 2024-08-25T21:09:39.464 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120894.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120894.log.gz 2024-08-25T21:09:39.465 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118686.log 2024-08-25T21:09:39.465 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40804.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40804.log.gz 2024-08-25T21:09:39.465 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100734.log 2024-08-25T21:09:39.465 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118686.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118686.log.gz 2024-08-25T21:09:39.465 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40465.log 2024-08-25T21:09:39.466 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100734.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100734.log.gz 2024-08-25T21:09:39.466 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24791.log 2024-08-25T21:09:39.466 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40465.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40465.log.gz 2024-08-25T21:09:39.466 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88039.log 2024-08-25T21:09:39.467 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24791.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.21848.log 2024-08-25T21:09:39.467 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24791.log.gz 2024-08-25T21:09:39.468 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88039.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88039.log.gz 2024-08-25T21:09:39.468 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38745.log 2024-08-25T21:09:39.468 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.21848.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.21848.log.gz 2024-08-25T21:09:39.469 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72488.log 2024-08-25T21:09:39.469 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38745.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38745.log.gz 2024-08-25T21:09:39.469 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58037.log 2024-08-25T21:09:39.470 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72488.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60084.log 2024-08-25T21:09:39.470 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72488.log.gz 2024-08-25T21:09:39.471 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.58037.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.58037.log.gz -- 2024-08-25T21:09:39.471 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.23623.log 2024-08-25T21:09:39.472 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60084.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60084.log.gz 2024-08-25T21:09:39.472 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90459.log 2024-08-25T21:09:39.472 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23623.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23623.log.gz 2024-08-25T21:09:39.473 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122993.log 2024-08-25T21:09:39.473 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90459.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90459.log.gz 2024-08-25T21:09:39.473 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98342.log 2024-08-25T21:09:39.474 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122993.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106086.log 2024-08-25T21:09:39.475 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122993.log.gz 2024-08-25T21:09:39.475 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98342.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22142.log 2024-08-25T21:09:39.475 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98342.log.gz 2024-08-25T21:09:39.476 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.106086.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106086.log.gz 2024-08-25T21:09:39.476 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81215.log 2024-08-25T21:09:39.477 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22142.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22142.log.gz 2024-08-25T21:09:39.477 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64995.log 2024-08-25T21:09:39.477 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.81215.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81215.log.gz 2024-08-25T21:09:39.477 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26168.log 2024-08-25T21:09:39.478 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64995.log.gz 2024-08-25T21:09:39.478 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56887.log 2024-08-25T21:09:39.479 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26168.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.26168.log.gz 2024-08-25T21:09:39.479 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29279.log 2024-08-25T21:09:39.479 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56887.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56887.log.gz 2024-08-25T21:09:39.479 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57177.log 2024-08-25T21:09:39.480 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29279.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29279.log.gz -5 2024-08-25T21:09:39.480 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.131452.log 2024-08-25T21:09:39.481 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57177.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57177.log.gz 2024-08-25T21:09:39.481 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121081.log 2024-08-25T21:09:39.481 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131452.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131452.log.gz 2024-08-25T21:09:39.481 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87934.log 2024-08-25T21:09:39.482 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121081.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121081.log.gz 2024-08-25T21:09:39.482 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136501.log 2024-08-25T21:09:39.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87934.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87934.log.gz 2024-08-25T21:09:39.483 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132648.log 2024-08-25T21:09:39.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136501.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136501.log.gz 2024-08-25T21:09:39.484 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100485.log 2024-08-25T21:09:39.484 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132648.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132648.log.gzgzip 2024-08-25T21:09:39.484 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134970.log 2024-08-25T21:09:39.485 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100485.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100485.log.gz 2024-08-25T21:09:39.485 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28981.log 2024-08-25T21:09:39.486 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134970.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134970.log.gz 2024-08-25T21:09:39.486 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61821.log 2024-08-25T21:09:39.487 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28981.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83338.log 2024-08-25T21:09:39.487 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28981.log.gz 2024-08-25T21:09:39.487 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61821.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61821.log.gz 2024-08-25T21:09:39.488 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117277.log 2024-08-25T21:09:39.488 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83338.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83338.log.gz 2024-08-25T21:09:39.488 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57778.log 2024-08-25T21:09:39.489 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.117277.log: gzip 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117277.log.gz -5 2024-08-25T21:09:39.489 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.96554.log 2024-08-25T21:09:39.489 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57778.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57778.log.gz 2024-08-25T21:09:39.489 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65496.log 2024-08-25T21:09:39.490 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96554.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96554.log.gz 2024-08-25T21:09:39.490 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.23061.log 2024-08-25T21:09:39.491 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65496.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28317.log 2024-08-25T21:09:39.491 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65496.log.gz 2024-08-25T21:09:39.492 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.23061.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.23061.log.gz 2024-08-25T21:09:39.492 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.32851.log 2024-08-25T21:09:39.492 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28317.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28317.log.gz 2024-08-25T21:09:39.493 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36796.log 2024-08-25T21:09:39.493 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.32851.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32851.log.gz 2024-08-25T21:09:39.493 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73495.log 2024-08-25T21:09:39.494 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36796.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36796.log.gz 2024-08-25T21:09:39.494 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42357.log 2024-08-25T21:09:39.495 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73495.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73495.log.gz 2024-08-25T21:09:39.495 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104886.log 2024-08-25T21:09:39.495 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42357.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42357.log.gz 2024-08-25T21:09:39.496 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30790.log 2024-08-25T21:09:39.496 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104886.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104886.log.gz 2024-08-25T21:09:39.497 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27365.log 2024-08-25T21:09:39.497 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30790.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30790.log.gz 2024-08-25T21:09:39.497 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120499.log 2024-08-25T21:09:39.498 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27365.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27365.log.gz 2024-08-25T21:09:39.498 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.79954.log 2024-08-25T21:09:39.499 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120499.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120499.log.gz 2024-08-25T21:09:39.499 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126405.log 2024-08-25T21:09:39.499 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.79954.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.79954.log.gz 2024-08-25T21:09:39.499 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99390.log 2024-08-25T21:09:39.500 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126405.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27723.log 2024-08-25T21:09:39.500 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126405.log.gz 2024-08-25T21:09:39.501 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99390.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99390.log.gz 2024-08-25T21:09:39.501 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130248.log 2024-08-25T21:09:39.502 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27723.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.27723.log.gz 2024-08-25T21:09:39.502 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75472.log 2024-08-25T21:09:39.502 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130248.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130248.log.gz 2024-08-25T21:09:39.503 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49512.log 2024-08-25T21:09:39.503 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75472.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75472.log.gz 2024-08-25T21:09:39.504 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120766.log 2024-08-25T21:09:39.504 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49512.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49512.log.gz 2024-08-25T21:09:39.504 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.45971.log 2024-08-25T21:09:39.505 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120766.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120766.log.gz 2024-08-25T21:09:39.505 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98255.log 2024-08-25T21:09:39.506 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.45971.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.45971.log.gz 2024-08-25T21:09:39.506 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108835.log 2024-08-25T21:09:39.507 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98255.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98255.log.gzgzip 2024-08-25T21:09:39.507 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86440.log 2024-08-25T21:09:39.507 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108835.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108835.log.gz 2024-08-25T21:09:39.508 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34914.log 2024-08-25T21:09:39.508 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86440.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40717.log 2024-08-25T21:09:39.508 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86440.log.gz 2024-08-25T21:09:39.509 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34914.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34914.log.gz 2024-08-25T21:09:39.509 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46386.log 2024-08-25T21:09:39.510 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40717.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40717.log.gz 2024-08-25T21:09:39.510 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.142591.log 2024-08-25T21:09:39.511 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46386.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46386.log.gz 2024-08-25T21:09:39.511 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140261.log 2024-08-25T21:09:39.511 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.142591.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.142591.log.gz 2024-08-25T21:09:39.512 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71873.log 2024-08-25T21:09:39.512 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140261.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140261.log.gz 2024-08-25T21:09:39.512 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87471.log 2024-08-25T21:09:39.513 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71873.log.gz 2024-08-25T21:09:39.513 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105165.log 2024-08-25T21:09:39.514 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87471.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87471.log.gz 2024-08-25T21:09:39.514 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133509.log 2024-08-25T21:09:39.514 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105165.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.admin.105165.log.gz 2024-08-25T21:09:39.515 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41139.log 2024-08-25T21:09:39.515 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133509.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133509.log.gz 2024-08-25T21:09:39.515 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88796.log 2024-08-25T21:09:39.516 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41139.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41139.log.gz 2024-08-25T21:09:39.516 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25509.log 2024-08-25T21:09:39.517 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88796.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88796.log.gz 2024-08-25T21:09:39.517 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68673.log 2024-08-25T21:09:39.517 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25509.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.25509.log.gz --verbose 2024-08-25T21:09:39.517 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.25502.log 2024-08-25T21:09:39.518 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68673.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68673.log.gz 2024-08-25T21:09:39.518 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106825.log 2024-08-25T21:09:39.519 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25502.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.25502.log.gz --verbose 2024-08-25T21:09:39.519 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.131850.log 2024-08-25T21:09:39.520 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106825.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106825.log.gz 2024-08-25T21:09:39.520 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137115.log 2024-08-25T21:09:39.520 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131850.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131850.log.gz 2024-08-25T21:09:39.521 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127318.log 2024-08-25T21:09:39.521 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137115.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74148.log 2024-08-25T21:09:39.521 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137115.log.gz 2024-08-25T21:09:39.522 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127318.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127318.log.gz 2024-08-25T21:09:39.522 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91248.log 2024-08-25T21:09:39.523 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74148.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74148.log.gz 2024-08-25T21:09:39.523 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133551.log 2024-08-25T21:09:39.523 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91248.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91248.log.gz 2024-08-25T21:09:39.524 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51612.log 2024-08-25T21:09:39.524 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133551.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133551.log.gz 2024-08-25T21:09:39.524 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109676.log 2024-08-25T21:09:39.525 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51612.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.51612.log.gz 2024-08-25T21:09:39.525 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114324.log 2024-08-25T21:09:39.526 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109676.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107284.log 2024-08-25T21:09:39.526 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109676.log.gz 2024-08-25T21:09:39.526 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114324.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131404.log 2024-08-25T21:09:39.526 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114324.log.gz 2024-08-25T21:09:39.527 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107284.log.gz 2024-08-25T21:09:39.527 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74002.log 2024-08-25T21:09:39.528 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131404.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41685.log 2024-08-25T21:09:39.528 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131404.log.gz 2024-08-25T21:09:39.529 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74002.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74002.log.gz 2024-08-25T21:09:39.529 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34625.log 2024-08-25T21:09:39.530 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41685.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41685.log.gz 2024-08-25T21:09:39.530 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51505.log 2024-08-25T21:09:39.531 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34625.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34625.log.gz 2024-08-25T21:09:39.531 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136594.log 2024-08-25T21:09:39.532 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51505.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51505.log.gz 2024-08-25T21:09:39.532 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41482.log 2024-08-25T21:09:39.532 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136594.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136594.log.gz 2024-08-25T21:09:39.532 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138112.log 2024-08-25T21:09:39.533 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41482.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41482.log.gz 2024-08-25T21:09:39.533 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114529.log 2024-08-25T21:09:39.534 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138112.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138112.log.gz 2024-08-25T21:09:39.534 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103671.log 2024-08-25T21:09:39.535 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114529.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114529.log.gz 2024-08-25T21:09:39.535 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69461.log 2024-08-25T21:09:39.536 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103671.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103671.log.gz 2024-08-25T21:09:39.536 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133088.log 2024-08-25T21:09:39.537 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69461.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69461.log.gz 2024-08-25T21:09:39.537 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92333.log 2024-08-25T21:09:39.537 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133088.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133088.log.gz 2024-08-25T21:09:39.537 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125535.log 2024-08-25T21:09:39.538 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92333.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92333.log.gz 2024-08-25T21:09:39.538 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111702.log 2024-08-25T21:09:39.538 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125535.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125535.log.gz 2024-08-25T21:09:39.539 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130491.log 2024-08-25T21:09:39.539 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111702.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111702.log.gz 2024-08-25T21:09:39.539 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117505.log 2024-08-25T21:09:39.540 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130491.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130491.log.gz 2024-08-25T21:09:39.540 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52639.log 2024-08-25T21:09:39.541 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117505.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117505.log.gz 2024-08-25T21:09:39.541 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118816.log 2024-08-25T21:09:39.542 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52639.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52639.log.gz 2024-08-25T21:09:39.542 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87187.log 2024-08-25T21:09:39.542 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118816.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118816.log.gz 2024-08-25T21:09:39.543 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100018.log 2024-08-25T21:09:39.543 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87187.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87187.log.gz 2024-08-25T21:09:39.543 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117413.log 2024-08-25T21:09:39.544 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100018.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100018.log.gz 2024-08-25T21:09:39.544 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138028.log 2024-08-25T21:09:39.545 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117413.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117413.log.gz 2024-08-25T21:09:39.545 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72760.log 2024-08-25T21:09:39.545 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138028.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138028.log.gz 2024-08-25T21:09:39.546 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133361.log 2024-08-25T21:09:39.546 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72760.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72760.log.gz 2024-08-25T21:09:39.546 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141812.log 2024-08-25T21:09:39.547 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133361.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130020.log 2024-08-25T21:09:39.547 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133361.log.gz 2024-08-25T21:09:39.548 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.141812.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104475.log 2024-08-25T21:09:39.548 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141812.log.gz 2024-08-25T21:09:39.549 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130020.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36940.log 2024-08-25T21:09:39.549 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130020.log.gz 2024-08-25T21:09:39.550 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.104475.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104475.log.gz 2024-08-25T21:09:39.550 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33871.log 2024-08-25T21:09:39.550 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36940.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36940.log.gz 2024-08-25T21:09:39.550 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42701.log 2024-08-25T21:09:39.551 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33871.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33871.log.gz 2024-08-25T21:09:39.551 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.12693.log 2024-08-25T21:09:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42701.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42701.log.gz 2024-08-25T21:09:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.24175.log 2024-08-25T21:09:39.552 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/ceph.tmp-client.admin.12693.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.12693.log.gz 2024-08-25T21:09:39.553 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78529.log 2024-08-25T21:09:39.553 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.24175.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.24175.log.gz 2024-08-25T21:09:39.553 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29529.log 2024-08-25T21:09:39.554 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78529.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.93683.log 2024-08-25T21:09:39.554 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78529.log.gz 2024-08-25T21:09:39.554 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29529.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.29529.log.gz 2024-08-25T21:09:39.555 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136837.log 2024-08-25T21:09:39.555 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.93683.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.93683.log.gz 2024-08-25T21:09:39.555 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42405.log 2024-08-25T21:09:39.557 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136837.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136837.log.gz 2024-08-25T21:09:39.557 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29836.log 2024-08-25T21:09:39.557 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42405.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42405.log.gz 2024-08-25T21:09:39.557 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97099.log 2024-08-25T21:09:39.557 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29836.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29836.log.gz 2024-08-25T21:09:39.558 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31499.log 2024-08-25T21:09:39.559 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97099.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97099.log.gz 2024-08-25T21:09:39.559 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111976.log 2024-08-25T21:09:39.559 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31499.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31499.log.gz 2024-08-25T21:09:39.559 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122660.log 2024-08-25T21:09:39.560 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111976.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111976.log.gz 2024-08-25T21:09:39.560 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140053.log 2024-08-25T21:09:39.561 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122660.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122660.log.gz 2024-08-25T21:09:39.561 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40969.log 2024-08-25T21:09:39.561 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.140053.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140053.log.gz 2024-08-25T21:09:39.561 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94025.log 2024-08-25T21:09:39.562 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40969.log.gz 2024-08-25T21:09:39.562 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125705.log 2024-08-25T21:09:39.563 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94025.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94025.log.gz 2024-08-25T21:09:39.563 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112960.log 2024-08-25T21:09:39.564 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125705.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125705.log.gz 2024-08-25T21:09:39.564 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77298.log 2024-08-25T21:09:39.564 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112960.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112960.log.gz 2024-08-25T21:09:39.564 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116674.log 2024-08-25T21:09:39.565 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77298.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81155.log 2024-08-25T21:09:39.565 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77298.log.gz 2024-08-25T21:09:39.566 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.116674.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.116674.log.gz 2024-08-25T21:09:39.566 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138357.log 2024-08-25T21:09:39.567 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81155.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81155.log.gz 2024-08-25T21:09:39.567 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80396.log 2024-08-25T21:09:39.568 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138357.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138357.log.gz 2024-08-25T21:09:39.568 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35255.log 2024-08-25T21:09:39.568 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80396.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80396.log.gz 2024-08-25T21:09:39.568 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120175.log 2024-08-25T21:09:39.569 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35255.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35255.log.gz 2024-08-25T21:09:39.570 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41515.log 2024-08-25T21:09:39.570 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120175.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120175.log.gz 2024-08-25T21:09:39.570 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121924.log 2024-08-25T21:09:39.570 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41515.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41515.log.gz 2024-08-25T21:09:39.571 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80096.log 2024-08-25T21:09:39.571 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121924.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121924.log.gz 2024-08-25T21:09:39.571 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.130321.log 2024-08-25T21:09:39.572 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80096.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130134.log 2024-08-25T21:09:39.572 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80096.log.gz 2024-08-25T21:09:39.573 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.130321.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.130321.log.gz 2024-08-25T21:09:39.573 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60921.log 2024-08-25T21:09:39.574 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130134.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130134.log.gz 2024-08-25T21:09:39.574 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75304.log 2024-08-25T21:09:39.574 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60921.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60921.log.gz 2024-08-25T21:09:39.574 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.95492.log 2024-08-25T21:09:39.575 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75304.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75304.log.gz 2024-08-25T21:09:39.575 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75419.log 2024-08-25T21:09:39.576 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.95492.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95492.log.gz 2024-08-25T21:09:39.576 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87363.log 2024-08-25T21:09:39.577 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75419.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75419.log.gz 2024-08-25T21:09:39.577 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39331.log 2024-08-25T21:09:39.577 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87363.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87363.log.gz 2024-08-25T21:09:39.577 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118706.log 2024-08-25T21:09:39.578 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39331.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39331.log.gz 2024-08-25T21:09:39.578 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38704.log 2024-08-25T21:09:39.578 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118706.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118706.log.gz 2024-08-25T21:09:39.578 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132563.log 2024-08-25T21:09:39.579 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38704.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38704.log.gz 2024-08-25T21:09:39.579 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28563.log 2024-08-25T21:09:39.580 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132563.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132563.log.gz 2024-08-25T21:09:39.580 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107327.log 2024-08-25T21:09:39.581 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28563.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28563.log.gz 2024-08-25T21:09:39.581 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106060.log 2024-08-25T21:09:39.582 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.107327.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107327.log.gz 2024-08-25T21:09:39.582 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33709.log 2024-08-25T21:09:39.582 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106060.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106060.log.gz 2024-08-25T21:09:39.583 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62976.log 2024-08-25T21:09:39.583 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33709.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33709.log.gz 2024-08-25T21:09:39.583 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74358.log 2024-08-25T21:09:39.584 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62976.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62976.log.gz 2024-08-25T21:09:39.584 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116723.log 2024-08-25T21:09:39.585 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74358.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74358.log.gz 2024-08-25T21:09:39.585 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68233.log 2024-08-25T21:09:39.585 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116723.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116723.log.gz 2024-08-25T21:09:39.586 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26687.log 2024-08-25T21:09:39.586 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68233.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68233.log.gz 2024-08-25T21:09:39.587 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125052.log 2024-08-25T21:09:39.587 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26687.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26687.log.gz 2024-08-25T21:09:39.587 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102336.log 2024-08-25T21:09:39.588 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125052.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125052.log.gzgzip 2024-08-25T21:09:39.588 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.139055.log 2024-08-25T21:09:39.589 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102336.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102336.log.gz 2024-08-25T21:09:39.589 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.23157.log 2024-08-25T21:09:39.590 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139055.log.gz 2024-08-25T21:09:39.590 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109423.log 2024-08-25T21:09:39.591 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.23157.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.23157.log.gz 2024-08-25T21:09:39.591 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.142801.log 2024-08-25T21:09:39.591 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109423.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109423.log.gz 2024-08-25T21:09:39.591 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23954.log 2024-08-25T21:09:39.592 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.142801.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.142801.log.gz 2024-08-25T21:09:39.592 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49365.log 2024-08-25T21:09:39.593 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23954.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23954.log.gz 2024-08-25T21:09:39.593 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89362.log 2024-08-25T21:09:39.594 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49365.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49365.log.gz 2024-08-25T21:09:39.594 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76650.log 2024-08-25T21:09:39.594 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89362.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89362.log.gz 2024-08-25T21:09:39.594 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34417.log 2024-08-25T21:09:39.595 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76650.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76650.log.gz 2024-08-25T21:09:39.595 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81222.log 2024-08-25T21:09:39.596 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34417.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34417.log.gz 2024-08-25T21:09:39.596 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91905.log 2024-08-25T21:09:39.596 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.81222.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81222.log.gz 2024-08-25T21:09:39.596 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113316.log 2024-08-25T21:09:39.597 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.91905.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.91905.log.gz 2024-08-25T21:09:39.597 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121862.log 2024-08-25T21:09:39.598 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113316.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113316.log.gz 2024-08-25T21:09:39.598 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87744.log 2024-08-25T21:09:39.598 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121862.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121862.log.gz 2024-08-25T21:09:39.599 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91601.log 2024-08-25T21:09:39.599 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87744.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87744.log.gz 2024-08-25T21:09:39.599 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112772.log 2024-08-25T21:09:39.600 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91601.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91601.log.gz 2024-08-25T21:09:39.600 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113402.log 2024-08-25T21:09:39.601 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112772.log.gz 2024-08-25T21:09:39.601 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48368.log 2024-08-25T21:09:39.602 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113402.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113402.log.gz 2024-08-25T21:09:39.602 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26610.log 2024-08-25T21:09:39.603 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48368.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48368.log.gz 2024-08-25T21:09:39.603 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25860.log 2024-08-25T21:09:39.603 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26610.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26610.log.gzgzip 2024-08-25T21:09:39.603 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72904.log 2024-08-25T21:09:39.604 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25860.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25860.log.gz 2024-08-25T21:09:39.604 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126323.log 2024-08-25T21:09:39.605 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72904.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72904.log.gz 2024-08-25T21:09:39.605 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97036.log 2024-08-25T21:09:39.605 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.126323.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126323.log.gz 2024-08-25T21:09:39.605 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28019.log 2024-08-25T21:09:39.606 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97036.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97036.log.gz 2024-08-25T21:09:39.606 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57057.log 2024-08-25T21:09:39.607 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28019.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28019.log.gz 2024-08-25T21:09:39.607 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112454.log 2024-08-25T21:09:39.607 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57057.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.admin.57057.log.gz 2024-08-25T21:09:39.608 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130624.log 2024-08-25T21:09:39.608 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112454.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112454.log.gz 2024-08-25T21:09:39.608 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113825.log 2024-08-25T21:09:39.609 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130624.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81376.log 2024-08-25T21:09:39.609 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130624.log.gz 2024-08-25T21:09:39.610 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113825.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113825.log.gz 2024-08-25T21:09:39.610 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132772.log 2024-08-25T21:09:39.611 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81376.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81376.log.gz 2024-08-25T21:09:39.611 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42723.log 2024-08-25T21:09:39.611 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132772.log.gz 2024-08-25T21:09:39.611 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43962.log 2024-08-25T21:09:39.612 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.42723.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.42723.log.gz 2024-08-25T21:09:39.612 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63610.log 2024-08-25T21:09:39.613 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43962.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43962.log.gz 2024-08-25T21:09:39.613 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64111.log 2024-08-25T21:09:39.614 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63610.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63610.log.gz 2024-08-25T21:09:39.614 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99706.log 2024-08-25T21:09:39.614 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64111.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64111.log.gz 2024-08-25T21:09:39.615 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121042.log 2024-08-25T21:09:39.615 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99706.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99706.log.gz 2024-08-25T21:09:39.615 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83074.log 2024-08-25T21:09:39.616 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121042.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121042.log.gz 2024-08-25T21:09:39.616 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80053.log 2024-08-25T21:09:39.617 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83074.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83074.log.gz -5 2024-08-25T21:09:39.617 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.89129.log 2024-08-25T21:09:39.617 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80053.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80053.log.gz 2024-08-25T21:09:39.617 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140283.log 2024-08-25T21:09:39.618 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89129.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89129.log.gzgzip 2024-08-25T21:09:39.618 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134361.log 2024-08-25T21:09:39.619 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140283.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140283.log.gz 2024-08-25T21:09:39.619 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95007.log 2024-08-25T21:09:39.620 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134361.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134361.log.gz 2024-08-25T21:09:39.620 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87059.log 2024-08-25T21:09:39.621 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95007.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95007.log.gz 2024-08-25T21:09:39.621 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137321.log 2024-08-25T21:09:39.622 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87059.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87059.log.gz 2024-08-25T21:09:39.622 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27071.log 2024-08-25T21:09:39.622 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137321.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137321.log.gz 2024-08-25T21:09:39.622 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106716.log 2024-08-25T21:09:39.623 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27071.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27071.log.gz 2024-08-25T21:09:39.623 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102122.log 2024-08-25T21:09:39.624 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106716.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.32641.log 2024-08-25T21:09:39.624 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106716.log.gz 2024-08-25T21:09:39.624 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102122.log.gz 2024-08-25T21:09:39.625 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108714.log 2024-08-25T21:09:39.625 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.32641.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.32641.log.gz 2024-08-25T21:09:39.625 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134384.log 2024-08-25T21:09:39.626 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108714.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108714.log.gz 2024-08-25T21:09:39.626 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41524.log 2024-08-25T21:09:39.627 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134384.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134384.log.gz 2024-08-25T21:09:39.627 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38240.log 2024-08-25T21:09:39.628 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41524.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41524.log.gz 2024-08-25T21:09:39.628 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76962.log 2024-08-25T21:09:39.628 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38240.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38240.log.gz 2024-08-25T21:09:39.629 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108919.log 2024-08-25T21:09:39.629 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76962.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113504.log 2024-08-25T21:09:39.629 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76962.log.gz 2024-08-25T21:09:39.630 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108919.log.gz 2024-08-25T21:09:39.630 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32028.log 2024-08-25T21:09:39.631 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113504.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113504.log.gz 2024-08-25T21:09:39.631 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51152.log 2024-08-25T21:09:39.632 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32028.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32028.log.gz 2024-08-25T21:09:39.632 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130295.log 2024-08-25T21:09:39.632 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51152.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51152.log.gz 2024-08-25T21:09:39.632 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51109.log 2024-08-25T21:09:39.633 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.130295.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130295.log.gz 2024-08-25T21:09:39.633 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29362.log 2024-08-25T21:09:39.634 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51109.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107589.log 2024-08-25T21:09:39.634 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51109.log.gz 2024-08-25T21:09:39.635 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29362.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29362.log.gz 2024-08-25T21:09:39.635 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28102.log 2024-08-25T21:09:39.636 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107589.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107589.log.gz 2024-08-25T21:09:39.636 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32743.log 2024-08-25T21:09:39.636 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28102.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28102.log.gz 2024-08-25T21:09:39.636 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82336.log 2024-08-25T21:09:39.637 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32743.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32743.log.gz 2024-08-25T21:09:39.637 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37754.log 2024-08-25T21:09:39.638 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82336.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82336.log.gz 2024-08-25T21:09:39.638 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28816.log 2024-08-25T21:09:39.639 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37754.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37754.log.gz 2024-08-25T21:09:39.639 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94045.log 2024-08-25T21:09:39.640 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28816.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28816.log.gz 2024-08-25T21:09:39.640 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116479.log 2024-08-25T21:09:39.641 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94045.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94045.log.gz 2024-08-25T21:09:39.641 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101155.log 2024-08-25T21:09:39.641 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116479.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116479.log.gz 2024-08-25T21:09:39.641 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94440.log 2024-08-25T21:09:39.642 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101155.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101155.log.gz 2024-08-25T21:09:39.642 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81834.log 2024-08-25T21:09:39.643 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94440.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94440.log.gz 2024-08-25T21:09:39.643 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121732.log 2024-08-25T21:09:39.644 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.81834.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81834.log.gz 2024-08-25T21:09:39.644 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96282.log 2024-08-25T21:09:39.644 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121732.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.121732.log.gz --verbose 2024-08-25T21:09:39.644 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.admin.105718.log 2024-08-25T21:09:39.645 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96282.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96282.log.gz 2024-08-25T21:09:39.645 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87365.log 2024-08-25T21:09:39.646 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.105718.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105718.log.gz 2024-08-25T21:09:39.646 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135873.log 2024-08-25T21:09:39.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87365.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87365.log.gz 2024-08-25T21:09:39.647 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78055.log 2024-08-25T21:09:39.648 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135873.log.gz 2024-08-25T21:09:39.648 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53042.log 2024-08-25T21:09:39.649 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78055.log.gz 2024-08-25T21:09:39.649 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50856.log 2024-08-25T21:09:39.650 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53042.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53042.log.gz 2024-08-25T21:09:39.650 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87997.log 2024-08-25T21:09:39.650 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50856.log.gz 2024-08-25T21:09:39.651 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106402.log 2024-08-25T21:09:39.651 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87997.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87997.log.gz 2024-08-25T21:09:39.651 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39499.log 2024-08-25T21:09:39.652 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106402.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106402.log.gz 2024-08-25T21:09:39.652 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123246.log 2024-08-25T21:09:39.653 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39499.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39499.log.gz 2024-08-25T21:09:39.653 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96869.log 2024-08-25T21:09:39.654 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123246.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123246.log.gz 2024-08-25T21:09:39.654 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89134.log 2024-08-25T21:09:39.654 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96869.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96869.log.gz 2024-08-25T21:09:39.654 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80469.log 2024-08-25T21:09:39.655 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89134.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112183.log 2024-08-25T21:09:39.655 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89134.log.gz 2024-08-25T21:09:39.656 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80469.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80469.log.gz 2024-08-25T21:09:39.656 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58472.log 2024-08-25T21:09:39.657 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112183.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.84618.log -- replaced with /var/log/ceph/cluster1-client.mirror.112183.log.gz 2024-08-25T21:09:39.657 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:39.657 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58472.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58472.log.gz 2024-08-25T21:09:39.658 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52496.log 2024-08-25T21:09:39.658 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84618.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84618.log.gz 2024-08-25T21:09:39.658 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68040.log 2024-08-25T21:09:39.660 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52496.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83986.log 2024-08-25T21:09:39.662 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52496.log.gz 2024-08-25T21:09:39.662 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68040.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68040.log.gz 2024-08-25T21:09:39.662 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110136.log 2024-08-25T21:09:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83986.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83986.log.gz 2024-08-25T21:09:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81655.log 2024-08-25T21:09:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110136.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110136.log.gz 2024-08-25T21:09:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39878.log 2024-08-25T21:09:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81655.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81655.log.gz 2024-08-25T21:09:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39964.log 2024-08-25T21:09:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39878.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39878.log.gz 2024-08-25T21:09:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82605.log 2024-08-25T21:09:39.665 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39964.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.39964.log.gz 2024-08-25T21:09:39.666 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34987.log 2024-08-25T21:09:39.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82605.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82605.log.gz 2024-08-25T21:09:39.666 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104612.log 2024-08-25T21:09:39.667 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34987.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34987.log.gz 2024-08-25T21:09:39.667 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48803.log 2024-08-25T21:09:39.667 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.104612.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104612.log.gz 2024-08-25T21:09:39.668 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28061.log 2024-08-25T21:09:39.668 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48803.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48803.log.gz 2024-08-25T21:09:39.668 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124122.log 2024-08-25T21:09:39.668 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28061.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28061.log.gz 2024-08-25T21:09:39.669 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33989.log 2024-08-25T21:09:39.669 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124122.log.gz 2024-08-25T21:09:39.669 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100587.log 2024-08-25T21:09:39.669 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33989.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33989.log.gz 2024-08-25T21:09:39.669 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49953.log 2024-08-25T21:09:39.670 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100587.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.100587.log.gz -5 2024-08-25T21:09:39.670 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.113826.log 2024-08-25T21:09:39.671 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49953.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49953.log.gz 2024-08-25T21:09:39.671 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.17403.log 2024-08-25T21:09:39.672 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113826.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113826.log.gz 2024-08-25T21:09:39.672 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135495.log 2024-08-25T21:09:39.673 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.17403.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.17403.log.gz 2024-08-25T21:09:39.673 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88331.log 2024-08-25T21:09:39.673 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135495.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135495.log.gz 2024-08-25T21:09:39.673 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33529.log 2024-08-25T21:09:39.674 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88331.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88331.log.gz 2024-08-25T21:09:39.674 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50436.log 2024-08-25T21:09:39.675 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33529.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33529.log.gz 2024-08-25T21:09:39.676 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.32926.log 2024-08-25T21:09:39.676 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50436.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50436.log.gz 2024-08-25T21:09:39.676 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129584.log 2024-08-25T21:09:39.677 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.32926.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32926.log.gz 2024-08-25T21:09:39.677 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101406.log 2024-08-25T21:09:39.677 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129584.log.gz 2024-08-25T21:09:39.677 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77890.log 2024-08-25T21:09:39.678 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101406.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101406.log.gz 2024-08-25T21:09:39.678 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87262.log 2024-08-25T21:09:39.679 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77890.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77890.log.gz 2024-08-25T21:09:39.679 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56907.log 2024-08-25T21:09:39.679 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87262.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87262.log.gz 2024-08-25T21:09:39.680 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115975.log 2024-08-25T21:09:39.680 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.56907.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56907.log.gz 2024-08-25T21:09:39.680 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47673.log 2024-08-25T21:09:39.681 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115975.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115975.log.gz 2024-08-25T21:09:39.681 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65706.log 2024-08-25T21:09:39.682 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47673.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47673.log.gz 2024-08-25T21:09:39.682 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44621.log 2024-08-25T21:09:39.683 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65706.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65706.log.gz 2024-08-25T21:09:39.683 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54825.log 2024-08-25T21:09:39.683 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.44621.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44621.log.gz 2024-08-25T21:09:39.684 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79790.log 2024-08-25T21:09:39.684 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54825.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54825.log.gz 2024-08-25T21:09:39.684 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89237.log 2024-08-25T21:09:39.685 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79790.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79790.log.gz 2024-08-25T21:09:39.685 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38555.log 2024-08-25T21:09:39.686 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89237.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89237.log.gz 2024-08-25T21:09:39.686 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62178.log 2024-08-25T21:09:39.687 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38555.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38555.log.gz 2024-08-25T21:09:39.687 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115166.log 2024-08-25T21:09:39.687 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62178.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62178.log.gz 2024-08-25T21:09:39.687 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91017.log 2024-08-25T21:09:39.688 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115166.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115166.log.gz 2024-08-25T21:09:39.688 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112184.log 2024-08-25T21:09:39.689 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91017.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91017.log.gz 2024-08-25T21:09:39.689 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83400.log 2024-08-25T21:09:39.690 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112184.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112184.log.gz 2024-08-25T21:09:39.690 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135642.log 2024-08-25T21:09:39.690 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83400.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.83400.log.gz 2024-08-25T21:09:39.690 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.46246.log 2024-08-25T21:09:39.691 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135642.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135642.log.gz 2024-08-25T21:09:39.691 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45570.log 2024-08-25T21:09:39.692 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46246.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46246.log.gz 2024-08-25T21:09:39.692 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57882.log 2024-08-25T21:09:39.693 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.45570.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45570.log.gz 2024-08-25T21:09:39.693 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115542.log 2024-08-25T21:09:39.694 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.57882.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57882.log.gz 2024-08-25T21:09:39.694 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.128120.log 2024-08-25T21:09:39.694 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115542.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115542.log.gz 2024-08-25T21:09:39.695 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129142.log 2024-08-25T21:09:39.696 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.128120.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.128120.log.gz 2024-08-25T21:09:39.696 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30432.log 2024-08-25T21:09:39.696 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.129142.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129142.log.gz 2024-08-25T21:09:39.696 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41014.log 2024-08-25T21:09:39.697 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30432.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30432.log.gz 2024-08-25T21:09:39.697 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120497.log 2024-08-25T21:09:39.697 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41014.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41014.log.gz 2024-08-25T21:09:39.697 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128720.log 2024-08-25T21:09:39.698 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120497.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120497.log.gz 2024-08-25T21:09:39.698 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116199.log 2024-08-25T21:09:39.699 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128720.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128720.log.gz 2024-08-25T21:09:39.699 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104157.log 2024-08-25T21:09:39.700 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116199.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116199.log.gz 2024-08-25T21:09:39.700 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123370.log 2024-08-25T21:09:39.700 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.104157.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104157.log.gz 2024-08-25T21:09:39.701 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141262.log 2024-08-25T21:09:39.701 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123370.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123370.log.gz 2024-08-25T21:09:39.702 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51822.log 2024-08-25T21:09:39.702 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141262.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141262.log.gz 2024-08-25T21:09:39.702 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58607.log 2024-08-25T21:09:39.703 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51822.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51822.log.gz 2024-08-25T21:09:39.703 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111678.log 2024-08-25T21:09:39.704 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.58607.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58607.log.gz 2024-08-25T21:09:39.704 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120063.log 2024-08-25T21:09:39.704 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111678.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111678.log.gz 2024-08-25T21:09:39.705 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56811.log 2024-08-25T21:09:39.705 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120063.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120063.log.gz 2024-08-25T21:09:39.705 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112560.log 2024-08-25T21:09:39.706 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56811.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56811.log.gz 2024-08-25T21:09:39.706 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72759.log 2024-08-25T21:09:39.707 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112560.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112560.log.gz 2024-08-25T21:09:39.707 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30474.log 2024-08-25T21:09:39.708 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72759.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72759.log.gz 2024-08-25T21:09:39.708 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121311.log 2024-08-25T21:09:39.708 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30474.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30474.log.gz 2024-08-25T21:09:39.708 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139630.log 2024-08-25T21:09:39.709 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121311.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121311.log.gz 2024-08-25T21:09:39.709 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24221.log 2024-08-25T21:09:39.710 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.139630.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139630.log.gz 2024-08-25T21:09:39.710 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82772.log 2024-08-25T21:09:39.711 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24221.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24221.log.gz 2024-08-25T21:09:39.711 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38325.log 2024-08-25T21:09:39.711 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82772.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82772.log.gz 2024-08-25T21:09:39.712 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38155.log 2024-08-25T21:09:39.712 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38325.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.38325.log.gz -- 2024-08-25T21:09:39.712 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.26461.log 2024-08-25T21:09:39.713 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38155.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38155.log.gz 2024-08-25T21:09:39.713 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63018.log 2024-08-25T21:09:39.714 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26461.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72800.log 2024-08-25T21:09:39.714 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26461.log.gz 2024-08-25T21:09:39.714 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63018.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63018.log.gz 2024-08-25T21:09:39.715 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77321.log 2024-08-25T21:09:39.715 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72800.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72800.log.gz 2024-08-25T21:09:39.716 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72310.log 2024-08-25T21:09:39.716 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77321.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77321.log.gz 2024-08-25T21:09:39.716 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47321.log 2024-08-25T21:09:39.717 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72310.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72310.log.gz 2024-08-25T21:09:39.717 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89170.log 2024-08-25T21:09:39.718 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47321.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47321.log.gz -5 2024-08-25T21:09:39.718 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.46748.log 2024-08-25T21:09:39.718 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89170.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89170.log.gz 2024-08-25T21:09:39.719 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74037.log 2024-08-25T21:09:39.719 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46748.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46748.log.gz 2024-08-25T21:09:39.720 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38743.log 2024-08-25T21:09:39.720 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74037.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74037.log.gz 2024-08-25T21:09:39.720 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87849.log 2024-08-25T21:09:39.721 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38743.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38743.log.gz 2024-08-25T21:09:39.721 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55441.log 2024-08-25T21:09:39.722 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87849.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87849.log.gz 2024-08-25T21:09:39.722 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87447.log 2024-08-25T21:09:39.722 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55441.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.admin.55441.log.gz /var/log/ceph/cluster1-client.admin.81807.log 2024-08-25T21:09:39.723 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:39.723 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87447.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87447.log.gz 2024-08-25T21:09:39.723 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116277.log 2024-08-25T21:09:39.724 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.81807.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81807.log.gz -5 2024-08-25T21:09:39.724 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.30370.log 2024-08-25T21:09:39.725 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.116277.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116277.log.gz 2024-08-25T21:09:39.725 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138212.log 2024-08-25T21:09:39.725 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30370.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30370.log.gz 2024-08-25T21:09:39.726 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113277.log 2024-08-25T21:09:39.727 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138212.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138212.log.gz 2024-08-25T21:09:39.727 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.24675.log 2024-08-25T21:09:39.727 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113277.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113277.log.gz 2024-08-25T21:09:39.727 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58265.log 2024-08-25T21:09:39.728 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.24675.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.24675.log.gz 2024-08-25T21:09:39.728 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37862.log 2024-08-25T21:09:39.729 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.58265.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58265.log.gz 2024-08-25T21:09:39.729 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52831.log 2024-08-25T21:09:39.730 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37862.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37862.log.gz 2024-08-25T21:09:39.730 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105650.log 2024-08-25T21:09:39.730 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52831.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.52831.log.gz 2024-08-25T21:09:39.730 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27511.log 2024-08-25T21:09:39.731 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105650.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105650.log.gz 2024-08-25T21:09:39.731 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97421.log 2024-08-25T21:09:39.732 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27511.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27511.log.gz 2024-08-25T21:09:39.732 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42439.log 2024-08-25T21:09:39.733 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97421.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97421.log.gz 2024-08-25T21:09:39.733 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96344.log 2024-08-25T21:09:39.733 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42439.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42439.log.gz 2024-08-25T21:09:39.733 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36731.log 2024-08-25T21:09:39.734 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96344.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96344.log.gz 2024-08-25T21:09:39.734 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68369.log 2024-08-25T21:09:39.735 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36731.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36731.log.gz 2024-08-25T21:09:39.735 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66086.log 2024-08-25T21:09:39.736 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.68369.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.68369.log.gz 2024-08-25T21:09:39.736 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36508.log 2024-08-25T21:09:39.736 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66086.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66086.log.gz 2024-08-25T21:09:39.736 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86523.log 2024-08-25T21:09:39.737 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36508.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36508.log.gz 2024-08-25T21:09:39.737 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135072.log 2024-08-25T21:09:39.738 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86523.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86523.log.gz 2024-08-25T21:09:39.738 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.22754.log 2024-08-25T21:09:39.739 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135072.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135072.log.gz 2024-08-25T21:09:39.739 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91271.log 2024-08-25T21:09:39.739 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.22754.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.22754.log.gz 2024-08-25T21:09:39.740 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.131474.log 2024-08-25T21:09:39.740 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91271.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91271.log.gz 2024-08-25T21:09:39.740 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44251.log 2024-08-25T21:09:39.741 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.131474.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.131474.log.gz 2024-08-25T21:09:39.741 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52536.log 2024-08-25T21:09:39.742 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44251.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44251.log.gz 2024-08-25T21:09:39.742 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50268.log 2024-08-25T21:09:39.743 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52536.log.gz 2024-08-25T21:09:39.743 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49617.log 2024-08-25T21:09:39.743 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50268.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50268.log.gz 2024-08-25T21:09:39.744 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133612.log 2024-08-25T21:09:39.744 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49617.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.122802.log 2024-08-25T21:09:39.744 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.49617.log.gz 2024-08-25T21:09:39.745 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133612.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133612.log.gz 2024-08-25T21:09:39.745 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105230.log 2024-08-25T21:09:39.746 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122802.log.gz 2024-08-25T21:09:39.746 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99535.log 2024-08-25T21:09:39.746 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105230.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105230.log.gz 2024-08-25T21:09:39.746 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74001.log 2024-08-25T21:09:39.747 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99535.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.99535.log.gz -- 2024-08-25T21:09:39.747 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.110514.log 2024-08-25T21:09:39.748 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74001.log.gz 2024-08-25T21:09:39.748 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72633.log 2024-08-25T21:09:39.749 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110514.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110514.log.gz 2024-08-25T21:09:39.749 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77127.log 2024-08-25T21:09:39.750 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72633.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72633.log.gz 2024-08-25T21:09:39.750 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86988.log 2024-08-25T21:09:39.750 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77127.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.77127.log.gz -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49012.log 2024-08-25T21:09:39.751 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:39.751 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86988.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86988.log.gz 2024-08-25T21:09:39.751 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54133.log 2024-08-25T21:09:39.752 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49012.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49012.log.gz 2024-08-25T21:09:39.752 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43674.log 2024-08-25T21:09:39.753 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54133.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54133.log.gz 2024-08-25T21:09:39.753 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103862.log 2024-08-25T21:09:39.754 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43674.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102404.log 2024-08-25T21:09:39.754 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43674.log.gz 2024-08-25T21:09:39.754 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103862.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103862.log.gz 2024-08-25T21:09:39.755 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53336.log 2024-08-25T21:09:39.755 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102404.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102404.log.gz 2024-08-25T21:09:39.756 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77656.log 2024-08-25T21:09:39.756 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53336.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53336.log.gz 2024-08-25T21:09:39.756 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119326.log 2024-08-25T21:09:39.757 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77656.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77656.log.gzgzip 2024-08-25T21:09:39.757 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.44026.log 2024-08-25T21:09:39.758 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119326.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119326.log.gz 2024-08-25T21:09:39.758 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68491.log 2024-08-25T21:09:39.759 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44026.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44026.log.gz 2024-08-25T21:09:39.759 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48590.log 2024-08-25T21:09:39.759 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68491.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68491.log.gz 2024-08-25T21:09:39.760 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123730.log 2024-08-25T21:09:39.760 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48590.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.48590.log.gz -5 2024-08-25T21:09:39.760 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.47780.log 2024-08-25T21:09:39.761 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123730.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123730.log.gz 2024-08-25T21:09:39.761 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81642.log 2024-08-25T21:09:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.47780.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47780.log.gz 2024-08-25T21:09:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.84247.log 2024-08-25T21:09:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81642.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81642.log.gz 2024-08-25T21:09:39.763 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85118.log 2024-08-25T21:09:39.763 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.84247.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.84247.log.gz 2024-08-25T21:09:39.763 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134252.log 2024-08-25T21:09:39.764 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85118.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85118.log.gz 2024-08-25T21:09:39.764 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86045.log 2024-08-25T21:09:39.765 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134252.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134252.log.gzgzip 2024-08-25T21:09:39.765 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25420.log 2024-08-25T21:09:39.766 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86045.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86045.log.gz 2024-08-25T21:09:39.766 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74457.log 2024-08-25T21:09:39.766 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25420.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25420.log.gz 2024-08-25T21:09:39.767 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51906.log 2024-08-25T21:09:39.767 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74457.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74457.log.gz 2024-08-25T21:09:39.768 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27872.log 2024-08-25T21:09:39.768 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51906.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132711.log 2024-08-25T21:09:39.768 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51906.log.gz 2024-08-25T21:09:39.769 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27872.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27872.log.gz 2024-08-25T21:09:39.769 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42103.log 2024-08-25T21:09:39.770 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132711.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132711.log.gz 2024-08-25T21:09:39.770 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111554.log 2024-08-25T21:09:39.771 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42103.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42103.log.gz 2024-08-25T21:09:39.771 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125576.log 2024-08-25T21:09:39.771 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111554.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.94139.log -- replaced with /var/log/ceph/cluster1-client.mirror.111554.log.gz 2024-08-25T21:09:39.771 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:39.772 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125576.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125576.log.gz 2024-08-25T21:09:39.772 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76190.log 2024-08-25T21:09:39.773 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94139.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94139.log.gz 2024-08-25T21:09:39.773 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83136.log 2024-08-25T21:09:39.773 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76190.log.gz 2024-08-25T21:09:39.774 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76014.log 2024-08-25T21:09:39.774 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83136.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83136.log.gz 2024-08-25T21:09:39.774 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98129.log 2024-08-25T21:09:39.775 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76014.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76014.log.gz 2024-08-25T21:09:39.775 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46427.log 2024-08-25T21:09:39.776 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98129.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98129.log.gz 2024-08-25T21:09:39.776 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108669.log 2024-08-25T21:09:39.777 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.46427.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46427.log.gz 2024-08-25T21:09:39.777 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43960.log 2024-08-25T21:09:39.778 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108669.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108669.log.gz 2024-08-25T21:09:39.778 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97272.log 2024-08-25T21:09:39.778 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43960.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126392.log 2024-08-25T21:09:39.779 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43960.log.gz 2024-08-25T21:09:39.779 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97272.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97272.log.gz 2024-08-25T21:09:39.780 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61422.log 2024-08-25T21:09:39.781 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126392.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126392.log.gz 2024-08-25T21:09:39.781 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105485.log 2024-08-25T21:09:39.781 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61422.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61422.log.gz 2024-08-25T21:09:39.781 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70505.log 2024-08-25T21:09:39.781 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105485.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105485.log.gz 2024-08-25T21:09:39.782 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93890.log 2024-08-25T21:09:39.782 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70505.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70505.log.gz 2024-08-25T21:09:39.782 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112412.log 2024-08-25T21:09:39.783 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.93890.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123518.log 0.0% 2024-08-25T21:09:39.783 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.93890.log.gz 2024-08-25T21:09:39.784 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112412.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112412.log.gz 2024-08-25T21:09:39.784 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60259.log 2024-08-25T21:09:39.784 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123518.log.gz 2024-08-25T21:09:39.785 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112185.log 2024-08-25T21:09:39.785 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60259.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60259.log.gz 2024-08-25T21:09:39.785 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105552.log 2024-08-25T21:09:39.786 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112185.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112185.log.gz 2024-08-25T21:09:39.786 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74283.log 2024-08-25T21:09:39.787 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105552.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111814.log 2024-08-25T21:09:39.787 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105552.log.gz 2024-08-25T21:09:39.788 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74283.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74283.log.gz 2024-08-25T21:09:39.788 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130441.log 2024-08-25T21:09:39.789 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111814.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111814.log.gz 2024-08-25T21:09:39.789 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108249.log 2024-08-25T21:09:39.790 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.130441.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130441.log.gz 2024-08-25T21:09:39.790 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50604.log 2024-08-25T21:09:39.790 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108249.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108249.log.gz 2024-08-25T21:09:39.790 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.45761.log 2024-08-25T21:09:39.791 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50604.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50604.log.gz 2024-08-25T21:09:39.791 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26758.log 2024-08-25T21:09:39.792 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45761.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45761.log.gz 2024-08-25T21:09:39.792 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50690.log 2024-08-25T21:09:39.792 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26758.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129543.log 2024-08-25T21:09:39.792 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26758.log.gz 2024-08-25T21:09:39.793 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50690.log.gz 2024-08-25T21:09:39.793 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97753.log 2024-08-25T21:09:39.794 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129543.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129543.log.gz 2024-08-25T21:09:39.794 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57179.log 2024-08-25T21:09:39.795 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97753.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97753.log.gz 2024-08-25T21:09:39.795 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42273.log 2024-08-25T21:09:39.796 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57179.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57179.log.gz 2024-08-25T21:09:39.796 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53713.log 2024-08-25T21:09:39.797 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42273.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42273.log.gz 2024-08-25T21:09:39.797 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40297.log 2024-08-25T21:09:39.797 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53713.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53713.log.gz 2024-08-25T21:09:39.797 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33133.log 2024-08-25T21:09:39.798 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40297.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40297.log.gz 2024-08-25T21:09:39.798 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62113.log 2024-08-25T21:09:39.799 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33133.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33133.log.gz 2024-08-25T21:09:39.799 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60457.log 2024-08-25T21:09:39.800 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62113.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62113.log.gz 2024-08-25T21:09:39.800 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28624.log 2024-08-25T21:09:39.800 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60457.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60457.log.gz 2024-08-25T21:09:39.801 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31075.log 2024-08-25T21:09:39.802 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28624.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28624.log.gz 2024-08-25T21:09:39.802 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34511.log 2024-08-25T21:09:39.802 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31075.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31075.log.gz 2024-08-25T21:09:39.802 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118241.log 2024-08-25T21:09:39.806 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34511.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34511.log.gz 2024-08-25T21:09:39.806 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63584.log 2024-08-25T21:09:39.809 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118241.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118241.log.gz 2024-08-25T21:09:39.810 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97271.log 2024-08-25T21:09:39.810 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63584.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33334.log 2024-08-25T21:09:39.810 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63584.log.gz 2024-08-25T21:09:39.810 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97271.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91380.log 2024-08-25T21:09:39.810 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97271.log.gz 2024-08-25T21:09:39.810 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33334.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33334.log.gz 2024-08-25T21:09:39.810 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50329.log 2024-08-25T21:09:39.810 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91380.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91380.log.gz 2024-08-25T21:09:39.810 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97836.log 2024-08-25T21:09:39.810 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50329.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50329.log.gz 2024-08-25T21:09:39.811 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81587.log 2024-08-25T21:09:39.811 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97836.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97836.log.gz 2024-08-25T21:09:39.811 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.59295.log 2024-08-25T21:09:39.811 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81587.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81587.log.gz 2024-08-25T21:09:39.811 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131192.log 2024-08-25T21:09:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.59295.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59295.log.gz 2024-08-25T21:09:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50371.log 2024-08-25T21:09:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131192.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131192.log.gz 2024-08-25T21:09:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141224.log 2024-08-25T21:09:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50371.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50371.log.gz 2024-08-25T21:09:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107834.log 2024-08-25T21:09:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141224.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113317.log 2024-08-25T21:09:39.813 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141224.log.gz 2024-08-25T21:09:39.814 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107834.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107834.log.gz 2024-08-25T21:09:39.815 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100171.log 2024-08-25T21:09:39.815 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113317.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113317.log.gz 2024-08-25T21:09:39.815 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79283.log 2024-08-25T21:09:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100171.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100171.log.gz 2024-08-25T21:09:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63104.log 2024-08-25T21:09:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79283.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79283.log.gz 2024-08-25T21:09:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130793.log 2024-08-25T21:09:39.817 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63104.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95750.log 2024-08-25T21:09:39.817 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63104.log.gz 2024-08-25T21:09:39.817 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130793.log.gz 2024-08-25T21:09:39.818 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103670.log 2024-08-25T21:09:39.818 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95750.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95750.log.gz -5 --verbose 2024-08-25T21:09:39.818 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.admin.82491.log 2024-08-25T21:09:39.819 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103670.log.gz 2024-08-25T21:09:39.819 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100883.log 2024-08-25T21:09:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82491.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82491.log.gz 2024-08-25T21:09:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54624.log 2024-08-25T21:09:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100883.log.gz 2024-08-25T21:09:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25651.log 2024-08-25T21:09:39.822 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54624.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54624.log.gzgzip 2024-08-25T21:09:39.822 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.103450.log 2024-08-25T21:09:39.822 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25651.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25651.log.gz 2024-08-25T21:09:39.823 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30244.log 2024-08-25T21:09:39.823 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103450.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.74399.log 2024-08-25T21:09:39.823 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.103450.log.gz 2024-08-25T21:09:39.824 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30244.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30244.log.gz 2024-08-25T21:09:39.824 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101974.log 2024-08-25T21:09:39.825 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74399.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86108.log 2024-08-25T21:09:39.825 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74399.log.gz 2024-08-25T21:09:39.826 INFO:teuthology.orchestra.run.smithi119.stderr: 92.3% -- replaced with /var/log/ceph/cluster1.audit.log.gz 2024-08-25T21:09:39.827 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101974.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101974.log.gz 2024-08-25T21:09:39.827 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94253.log 2024-08-25T21:09:39.827 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86108.log.gz 2024-08-25T21:09:39.827 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23360.log 2024-08-25T21:09:39.827 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94253.log.gzgzip 2024-08-25T21:09:39.828 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41767.log 2024-08-25T21:09:39.828 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23360.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23360.log.gz 2024-08-25T21:09:39.828 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39208.log 2024-08-25T21:09:39.829 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41767.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86799.log 2024-08-25T21:09:39.829 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41767.log.gz 2024-08-25T21:09:39.830 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39208.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39208.log.gz 2024-08-25T21:09:39.830 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33183.log 2024-08-25T21:09:39.831 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86799.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86799.log.gz 2024-08-25T21:09:39.831 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101279.log 2024-08-25T21:09:39.832 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33183.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33183.log.gz 2024-08-25T21:09:39.832 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131602.log 2024-08-25T21:09:39.833 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101279.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84950.log 2024-08-25T21:09:39.833 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101279.log.gz 2024-08-25T21:09:39.833 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131602.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131602.log.gz 2024-08-25T21:09:39.833 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138966.log 2024-08-25T21:09:39.834 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84950.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84950.log.gz 2024-08-25T21:09:39.834 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.143011.log 2024-08-25T21:09:39.835 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138966.log.gz 2024-08-25T21:09:39.835 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78014.log 2024-08-25T21:09:39.836 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.143011.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120561.log 2024-08-25T21:09:39.836 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.143011.log.gz 2024-08-25T21:09:39.837 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78014.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78014.log.gz 2024-08-25T21:09:39.837 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116381.log 2024-08-25T21:09:39.838 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120561.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120561.log.gz 2024-08-25T21:09:39.838 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127589.log 2024-08-25T21:09:39.839 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116381.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116381.log.gz 2024-08-25T21:09:39.839 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70102.log 2024-08-25T21:09:39.839 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127589.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127589.log.gz 2024-08-25T21:09:39.840 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131194.log 2024-08-25T21:09:39.840 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70102.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139341.log 2024-08-25T21:09:39.840 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70102.log.gz 2024-08-25T21:09:39.841 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131194.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131194.log.gz 2024-08-25T21:09:39.841 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134654.log 2024-08-25T21:09:39.842 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139341.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139341.log.gz 2024-08-25T21:09:39.842 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141905.log 2024-08-25T21:09:39.843 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134654.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134654.log.gz 2024-08-25T21:09:39.843 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123412.log 2024-08-25T21:09:39.844 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141905.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141905.log.gz 2024-08-25T21:09:39.844 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101721.log 2024-08-25T21:09:39.844 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123412.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123412.log.gz 2024-08-25T21:09:39.845 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50898.log 2024-08-25T21:09:39.845 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101721.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101721.log.gz 2024-08-25T21:09:39.845 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98887.log 2024-08-25T21:09:39.846 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50898.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50898.log.gz 2024-08-25T21:09:39.846 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34074.log 2024-08-25T21:09:39.847 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98887.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98887.log.gz 2024-08-25T21:09:39.847 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63903.log 2024-08-25T21:09:39.848 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34074.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34074.log.gz 2024-08-25T21:09:39.848 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104412.log 2024-08-25T21:09:39.849 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63903.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63903.log.gz 2024-08-25T21:09:39.849 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37107.log 2024-08-25T21:09:39.849 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104412.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104412.log.gz 2024-08-25T21:09:39.850 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81693.log 2024-08-25T21:09:39.851 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37107.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37107.log.gz 2024-08-25T21:09:39.851 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78396.log 2024-08-25T21:09:39.851 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.81693.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81693.log.gz 2024-08-25T21:09:39.851 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48506.log 2024-08-25T21:09:39.853 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78396.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78396.log.gz 2024-08-25T21:09:39.853 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46406.log 2024-08-25T21:09:39.853 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48506.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48506.log.gz 2024-08-25T21:09:39.854 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67516.log 2024-08-25T21:09:39.854 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46406.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46406.log.gz 2024-08-25T21:09:39.854 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37401.log 2024-08-25T21:09:39.855 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67516.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67516.log.gz 2024-08-25T21:09:39.856 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55511.log 2024-08-25T21:09:39.856 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37401.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37401.log.gz 2024-08-25T21:09:39.857 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77470.log 2024-08-25T21:09:39.857 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55511.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55511.log.gz 2024-08-25T21:09:39.857 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73134.log 2024-08-25T21:09:39.858 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77470.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77470.log.gz 2024-08-25T21:09:39.858 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31428.log 2024-08-25T21:09:39.858 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73134.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73134.log.gz 2024-08-25T21:09:39.859 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56545.log 2024-08-25T21:09:39.859 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31428.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31428.log.gz 2024-08-25T21:09:39.860 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99184.log 2024-08-25T21:09:39.860 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56545.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56545.log.gz 2024-08-25T21:09:39.861 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.36572.log 2024-08-25T21:09:39.861 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99184.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99184.log.gz 2024-08-25T21:09:39.861 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83111.log 2024-08-25T21:09:39.862 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.36572.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.36572.log.gz 2024-08-25T21:09:39.862 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140291.log 2024-08-25T21:09:39.863 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83111.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83111.log.gz 2024-08-25T21:09:39.864 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131751.log 2024-08-25T21:09:39.864 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140291.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140291.log.gz 2024-08-25T21:09:39.864 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43740.log 2024-08-25T21:09:39.865 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131751.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131751.log.gz 2024-08-25T21:09:39.865 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69187.log 2024-08-25T21:09:39.866 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43740.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43740.log.gz 2024-08-25T21:09:39.866 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116517.log 2024-08-25T21:09:39.867 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69187.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69187.log.gz 2024-08-25T21:09:39.867 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81101.log 2024-08-25T21:09:39.868 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116517.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116517.log.gz 2024-08-25T21:09:39.868 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37316.log 2024-08-25T21:09:39.869 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81101.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81101.log.gz 2024-08-25T21:09:39.869 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97328.log 2024-08-25T21:09:39.870 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37316.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37316.log.gz 2024-08-25T21:09:39.870 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64279.log 2024-08-25T21:09:39.871 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97328.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97328.log.gz 2024-08-25T21:09:39.871 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108043.log 2024-08-25T21:09:39.872 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64279.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64279.log.gz 2024-08-25T21:09:39.872 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30561.log 2024-08-25T21:09:39.873 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108043.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108043.log.gz 2024-08-25T21:09:39.873 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85957.log 2024-08-25T21:09:39.873 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30561.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30561.log.gz 2024-08-25T21:09:39.874 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136921.log 2024-08-25T21:09:39.875 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85957.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85957.log.gz 2024-08-25T21:09:39.875 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68676.log 2024-08-25T21:09:39.876 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136921.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136921.log.gz 2024-08-25T21:09:39.876 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66908.log 2024-08-25T21:09:39.876 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68676.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68676.log.gz 2024-08-25T21:09:39.877 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78472.log 2024-08-25T21:09:39.877 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66908.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66908.log.gz 2024-08-25T21:09:39.878 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73708.log 2024-08-25T21:09:39.878 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78472.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78472.log.gz 2024-08-25T21:09:39.879 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114232.log 2024-08-25T21:09:39.880 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73708.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73708.log.gz 2024-08-25T21:09:39.880 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88986.log 2024-08-25T21:09:39.880 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114232.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114232.log.gz 2024-08-25T21:09:39.881 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37966.log 2024-08-25T21:09:39.881 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88986.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88986.log.gz 2024-08-25T21:09:39.882 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139627.log 2024-08-25T21:09:39.882 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37966.log.gz 2024-08-25T21:09:39.882 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89448.log 2024-08-25T21:09:39.883 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139627.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139627.log.gz 2024-08-25T21:09:39.883 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53797.log 2024-08-25T21:09:39.884 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89448.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89448.log.gz 2024-08-25T21:09:39.884 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32181.log 2024-08-25T21:09:39.885 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53797.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53797.log.gz 2024-08-25T21:09:39.885 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76139.log 2024-08-25T21:09:39.886 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32181.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32181.log.gz 2024-08-25T21:09:39.886 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28522.log 2024-08-25T21:09:39.887 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76139.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76139.log.gz 2024-08-25T21:09:39.887 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56395.log 2024-08-25T21:09:39.888 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28522.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28522.log.gz 2024-08-25T21:09:39.888 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.91927.log 2024-08-25T21:09:39.889 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.56395.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56395.log.gz 2024-08-25T21:09:39.889 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111170.log 2024-08-25T21:09:39.890 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.91927.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.91927.log.gz 2024-08-25T21:09:39.890 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28521.log 2024-08-25T21:09:39.891 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111170.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111170.log.gz 2024-08-25T21:09:39.891 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68674.log 2024-08-25T21:09:39.892 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28521.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28521.log.gz 2024-08-25T21:09:39.892 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51486.log 2024-08-25T21:09:39.893 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68674.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68674.log.gz 2024-08-25T21:09:39.893 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69536.log 2024-08-25T21:09:39.893 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51486.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51486.log.gz 2024-08-25T21:09:39.894 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99094.log 2024-08-25T21:09:39.894 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69536.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69536.log.gz 2024-08-25T21:09:39.895 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28204.log 2024-08-25T21:09:39.895 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99094.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99094.log.gz 2024-08-25T21:09:39.896 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70418.log 2024-08-25T21:09:39.896 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28204.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28204.log.gz 2024-08-25T21:09:39.897 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141211.log 2024-08-25T21:09:39.897 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70418.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70418.log.gz 2024-08-25T21:09:39.897 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106516.log 2024-08-25T21:09:39.898 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141211.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141211.log.gz 2024-08-25T21:09:39.899 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92834.log 2024-08-25T21:09:39.899 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106516.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106516.log.gz 2024-08-25T21:09:39.899 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46178.log 2024-08-25T21:09:39.900 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92834.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92834.log.gz 2024-08-25T21:09:39.901 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30242.log 2024-08-25T21:09:39.901 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46178.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46178.log.gz 2024-08-25T21:09:39.901 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35850.log 2024-08-25T21:09:39.902 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30242.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30242.log.gz 2024-08-25T21:09:39.902 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45147.log 2024-08-25T21:09:39.903 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35850.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35850.log.gz 2024-08-25T21:09:39.903 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87347.log 2024-08-25T21:09:39.904 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45147.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45147.log.gz 2024-08-25T21:09:39.904 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105463.log 2024-08-25T21:09:39.905 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87347.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87347.log.gz 2024-08-25T21:09:39.905 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51276.log 2024-08-25T21:09:39.906 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105463.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105463.log.gz 2024-08-25T21:09:39.906 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67760.log 2024-08-25T21:09:39.907 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51276.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51276.log.gz 2024-08-25T21:09:39.907 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128716.log 2024-08-25T21:09:39.908 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67760.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67760.log.gz 2024-08-25T21:09:39.908 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93265.log 2024-08-25T21:09:39.909 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128716.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128716.log.gz 2024-08-25T21:09:39.909 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141517.log 2024-08-25T21:09:39.909 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.93265.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93265.log.gz 2024-08-25T21:09:39.910 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25001.log 2024-08-25T21:09:39.911 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141517.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141517.log.gz 2024-08-25T21:09:39.911 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69065.log 2024-08-25T21:09:39.912 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25001.log.gz 2024-08-25T21:09:39.912 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112413.log 2024-08-25T21:09:39.912 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69065.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69065.log.gz 2024-08-25T21:09:39.913 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115100.log 2024-08-25T21:09:39.913 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112413.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112413.log.gz 2024-08-25T21:09:39.914 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126041.log 2024-08-25T21:09:39.915 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115100.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115100.log.gz 2024-08-25T21:09:39.915 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50815.log 2024-08-25T21:09:39.915 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.126041.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126041.log.gz 2024-08-25T21:09:39.916 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69136.log 2024-08-25T21:09:39.916 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50815.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50815.log.gz 2024-08-25T21:09:39.917 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128399.log 2024-08-25T21:09:39.917 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.69136.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69136.log.gz 2024-08-25T21:09:39.917 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65118.log 2024-08-25T21:09:39.918 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128399.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128399.log.gz 2024-08-25T21:09:39.919 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54586.log 2024-08-25T21:09:39.920 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65118.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96868.log 2024-08-25T21:09:39.920 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65118.log.gz 2024-08-25T21:09:39.921 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54586.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54586.log.gz 2024-08-25T21:09:39.921 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123522.log 2024-08-25T21:09:39.921 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96868.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96868.log.gz 2024-08-25T21:09:39.921 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77530.log 2024-08-25T21:09:39.922 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123522.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123522.log.gz 2024-08-25T21:09:39.922 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35766.log 2024-08-25T21:09:39.923 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77530.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77530.log.gz 2024-08-25T21:09:39.923 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59780.log 2024-08-25T21:09:39.924 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.35766.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35766.log.gz 2024-08-25T21:09:39.924 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25933.log 2024-08-25T21:09:39.925 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59780.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59780.log.gz 2024-08-25T21:09:39.925 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70532.log 2024-08-25T21:09:39.926 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25933.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25933.log.gz 2024-08-25T21:09:39.926 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74673.log 2024-08-25T21:09:39.926 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70532.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70532.log.gz 2024-08-25T21:09:39.927 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111074.log 2024-08-25T21:09:39.927 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74673.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74673.log.gz 2024-08-25T21:09:39.927 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50900.log 2024-08-25T21:09:39.928 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111074.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111074.log.gz 2024-08-25T21:09:39.928 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37820.log 2024-08-25T21:09:39.929 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50900.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50900.log.gz 2024-08-25T21:09:39.929 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66871.log 2024-08-25T21:09:39.930 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37820.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37820.log.gz 2024-08-25T21:09:39.931 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.22805.log 2024-08-25T21:09:39.931 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66871.log.gz 2024-08-25T21:09:39.932 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101680.log 2024-08-25T21:09:39.932 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.22805.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.22805.log.gz 2024-08-25T21:09:39.933 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24131.log 2024-08-25T21:09:39.933 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101680.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101680.log.gz 2024-08-25T21:09:39.933 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98969.log 2024-08-25T21:09:39.934 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24131.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24131.log.gz 2024-08-25T21:09:39.934 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127749.log 2024-08-25T21:09:39.935 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98969.log.gz 2024-08-25T21:09:39.935 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125115.log 2024-08-25T21:09:39.936 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127749.log.gz 2024-08-25T21:09:39.936 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113650.log 2024-08-25T21:09:39.937 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125115.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125115.log.gz 2024-08-25T21:09:39.937 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60877.log 2024-08-25T21:09:39.938 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113650.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113650.log.gz 2024-08-25T21:09:39.938 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108920.log 2024-08-25T21:09:39.939 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60877.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73198.log 2024-08-25T21:09:39.939 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60877.log.gz 2024-08-25T21:09:39.940 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108920.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108920.log.gz 2024-08-25T21:09:39.940 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67536.log 2024-08-25T21:09:39.941 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73198.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73198.log.gz 2024-08-25T21:09:39.941 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133463.log 2024-08-25T21:09:39.942 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67536.log.gz 2024-08-25T21:09:39.942 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36336.log 2024-08-25T21:09:39.943 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133463.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133463.log.gz 2024-08-25T21:09:39.943 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30048.log 2024-08-25T21:09:39.944 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36336.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36336.log.gz 2024-08-25T21:09:39.944 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87516.log 2024-08-25T21:09:39.945 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30048.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30048.log.gz 2024-08-25T21:09:39.945 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97627.log 2024-08-25T21:09:39.946 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87516.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87516.log.gz 2024-08-25T21:09:39.946 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115738.log 2024-08-25T21:09:39.947 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97627.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97627.log.gz 2024-08-25T21:09:39.947 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75767.log 2024-08-25T21:09:39.948 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115738.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139165.log 2024-08-25T21:09:39.948 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115738.log.gz 2024-08-25T21:09:39.949 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75767.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75767.log.gz 2024-08-25T21:09:39.949 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.32615.log 2024-08-25T21:09:39.950 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139165.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139165.log.gz 2024-08-25T21:09:39.950 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106753.log 2024-08-25T21:09:39.951 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.32615.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32615.log.gz 2024-08-25T21:09:39.951 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54862.log 2024-08-25T21:09:39.952 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106753.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86231.log 2024-08-25T21:09:39.953 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106753.log.gz 2024-08-25T21:09:39.953 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54862.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54862.log.gz 2024-08-25T21:09:39.954 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63438.log 2024-08-25T21:09:39.955 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86231.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71720.log 2024-08-25T21:09:39.955 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86231.log.gz 2024-08-25T21:09:39.956 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63438.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63438.log.gz 2024-08-25T21:09:39.956 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57713.log 2024-08-25T21:09:39.957 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71720.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46018.log 2024-08-25T21:09:39.957 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71720.log.gz 2024-08-25T21:09:39.958 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57713.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57713.log.gz 2024-08-25T21:09:39.958 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131432.log 2024-08-25T21:09:39.959 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46018.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46018.log.gz 2024-08-25T21:09:39.959 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120560.log 2024-08-25T21:09:39.960 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131432.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131432.log.gz 2024-08-25T21:09:39.960 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141470.log 2024-08-25T21:09:39.961 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120560.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120560.log.gz 2024-08-25T21:09:39.961 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99995.log 2024-08-25T21:09:39.962 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.141470.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141470.log.gz 2024-08-25T21:09:39.962 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94709.log 2024-08-25T21:09:39.963 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99995.log.gz 2024-08-25T21:09:39.963 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135872.log 2024-08-25T21:09:39.964 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94709.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94709.log.gz 2024-08-25T21:09:39.964 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.119749.log 2024-08-25T21:09:39.965 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135872.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135872.log.gz 2024-08-25T21:09:39.965 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86486.log 2024-08-25T21:09:39.966 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.119749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.119749.log.gz 2024-08-25T21:09:39.966 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53544.log 2024-08-25T21:09:39.967 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86486.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86486.log.gz 2024-08-25T21:09:39.967 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86859.log 2024-08-25T21:09:39.968 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53544.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53544.log.gz 2024-08-25T21:09:39.968 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119043.log 2024-08-25T21:09:39.969 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86859.log.gz 2024-08-25T21:09:39.969 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136715.log 2024-08-25T21:09:39.970 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119043.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119043.log.gz 2024-08-25T21:09:39.971 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97060.log 2024-08-25T21:09:39.971 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136715.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136715.log.gz 2024-08-25T21:09:39.971 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73240.log 2024-08-25T21:09:39.972 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97060.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97060.log.gz 2024-08-25T21:09:39.973 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122407.log 2024-08-25T21:09:39.973 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73240.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73240.log.gz 2024-08-25T21:09:39.974 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85346.log 2024-08-25T21:09:39.974 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122407.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122407.log.gz 2024-08-25T21:09:39.975 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118381.log 2024-08-25T21:09:39.975 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85346.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85346.log.gz 2024-08-25T21:09:39.976 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87227.log 2024-08-25T21:09:39.976 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.118381.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118381.log.gz 2024-08-25T21:09:39.977 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52494.log 2024-08-25T21:09:39.977 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87227.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87227.log.gz 2024-08-25T21:09:39.978 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48802.log 2024-08-25T21:09:39.978 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52494.log.gz 2024-08-25T21:09:39.978 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.60211.log 2024-08-25T21:09:39.979 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48802.log.gz 2024-08-25T21:09:39.979 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129497.log 2024-08-25T21:09:39.980 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.60211.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134382.log 2024-08-25T21:09:39.980 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.60211.log.gz 2024-08-25T21:09:39.981 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129497.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129497.log.gz 2024-08-25T21:09:39.981 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79548.log 2024-08-25T21:09:39.982 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134382.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134382.log.gz 2024-08-25T21:09:39.982 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116606.log 2024-08-25T21:09:39.982 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79548.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79548.log.gz 2024-08-25T21:09:39.983 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78601.log 2024-08-25T21:09:39.983 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.116606.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116606.log.gz 2024-08-25T21:09:39.983 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75369.log 2024-08-25T21:09:39.984 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78601.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68895.log 2024-08-25T21:09:39.984 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78601.log.gz 2024-08-25T21:09:39.985 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75369.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75369.log.gz 2024-08-25T21:09:39.986 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74844.log 2024-08-25T21:09:39.986 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.68895.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.68895.log.gz 2024-08-25T21:09:39.986 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34941.log 2024-08-25T21:09:39.987 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74844.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74844.log.gz 2024-08-25T21:09:39.987 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102014.log 2024-08-25T21:09:39.987 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34941.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34941.log.gz 2024-08-25T21:09:39.988 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55312.log 2024-08-25T21:09:39.989 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102014.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102014.log.gz 2024-08-25T21:09:39.989 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38788.log 2024-08-25T21:09:39.989 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55312.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55312.log.gz 2024-08-25T21:09:39.989 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91381.log 2024-08-25T21:09:39.990 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38788.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38788.log.gz 2024-08-25T21:09:39.990 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125575.log 2024-08-25T21:09:39.991 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91381.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91381.log.gz 2024-08-25T21:09:39.991 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134863.log 2024-08-25T21:09:39.991 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125575.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125575.log.gz 2024-08-25T21:09:39.991 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.57829.log 2024-08-25T21:09:39.992 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134863.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134863.log.gz 2024-08-25T21:09:39.992 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74127.log 2024-08-25T21:09:39.993 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.57829.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.57829.log.gz 2024-08-25T21:09:39.993 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73371.log 2024-08-25T21:09:39.994 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74127.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74127.log.gz 2024-08-25T21:09:39.994 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138294.log 2024-08-25T21:09:39.994 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73371.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73371.log.gz 2024-08-25T21:09:39.995 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108071.log 2024-08-25T21:09:39.995 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138294.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138294.log.gz 2024-08-25T21:09:39.995 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94595.log 2024-08-25T21:09:39.997 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108071.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108071.log.gz 2024-08-25T21:09:39.997 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83948.log 2024-08-25T21:09:39.997 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94595.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94595.log.gz 2024-08-25T21:09:39.997 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35877.log 2024-08-25T21:09:39.998 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83948.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83948.log.gz 2024-08-25T21:09:39.998 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62599.log 2024-08-25T21:09:39.999 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35877.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35877.log.gz 2024-08-25T21:09:39.999 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28435.log 2024-08-25T21:09:40.000 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62599.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62599.log.gz 2024-08-25T21:09:40.000 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120645.log 2024-08-25T21:09:40.000 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28435.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28435.log.gz 2024-08-25T21:09:40.000 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59663.log 2024-08-25T21:09:40.001 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120645.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120645.log.gz 2024-08-25T21:09:40.002 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140373.log 2024-08-25T21:09:40.002 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59663.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59663.log.gz 2024-08-25T21:09:40.002 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106583.log 2024-08-25T21:09:40.003 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.140373.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140373.log.gz 2024-08-25T21:09:40.003 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.127985.log 2024-08-25T21:09:40.004 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106583.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106583.log.gz 2024-08-25T21:09:40.004 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87159.log 2024-08-25T21:09:40.004 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.127985.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.127985.log.gz 2024-08-25T21:09:40.004 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118629.log 2024-08-25T21:09:40.005 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87159.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87159.log.gz 2024-08-25T21:09:40.005 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133322.log 2024-08-25T21:09:40.006 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.118629.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118629.log.gz 2024-08-25T21:09:40.006 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111911.log 2024-08-25T21:09:40.007 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133322.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133322.log.gz 2024-08-25T21:09:40.007 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27932.log 2024-08-25T21:09:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111911.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111911.log.gz 2024-08-25T21:09:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100484.log 2024-08-25T21:09:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27932.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27932.log.gz 2024-08-25T21:09:40.008 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25315.log 2024-08-25T21:09:40.009 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100484.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.100484.log.gz -- 2024-08-25T21:09:40.009 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.30243.log 2024-08-25T21:09:40.009 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25315.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25315.log.gz 2024-08-25T21:09:40.010 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94273.log 2024-08-25T21:09:40.010 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30243.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108567.log 2024-08-25T21:09:40.010 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30243.log.gz 2024-08-25T21:09:40.011 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94273.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94273.log.gz 2024-08-25T21:09:40.011 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80305.log 2024-08-25T21:09:40.012 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108567.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108567.log.gz 2024-08-25T21:09:40.012 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82263.log 2024-08-25T21:09:40.013 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80305.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80305.log.gz 2024-08-25T21:09:40.013 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22873.log 2024-08-25T21:09:40.013 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82263.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.80150.log 2024-08-25T21:09:40.014 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.82263.log.gz 2024-08-25T21:09:40.014 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22873.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60295.log 2024-08-25T21:09:40.014 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22873.log.gz 2024-08-25T21:09:40.015 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80150.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80150.log.gz 2024-08-25T21:09:40.015 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123855.log 2024-08-25T21:09:40.016 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60295.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111513.log 2024-08-25T21:09:40.016 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60295.log.gz 2024-08-25T21:09:40.017 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123855.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123855.log.gz 2024-08-25T21:09:40.017 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82316.log 2024-08-25T21:09:40.017 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111513.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111513.log.gz 2024-08-25T21:09:40.018 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66904.log 2024-08-25T21:09:40.018 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82316.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82316.log.gz 2024-08-25T21:09:40.018 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108248.log 2024-08-25T21:09:40.019 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66904.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112247.log 2024-08-25T21:09:40.019 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66904.log.gz 2024-08-25T21:09:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108248.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108248.log.gz 2024-08-25T21:09:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124335.log 2024-08-25T21:09:40.020 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112247.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.112247.log.gz -- /var/log/ceph/cluster1-client.mirror.110161.log 2024-08-25T21:09:40.021 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:40.021 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124335.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124335.log.gz 2024-08-25T21:09:40.022 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124513.log 2024-08-25T21:09:40.022 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110161.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44883.log 2024-08-25T21:09:40.022 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110161.log.gz 2024-08-25T21:09:40.023 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124513.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137805.log 2024-08-25T21:09:40.024 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124513.log.gz 2024-08-25T21:09:40.024 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44883.log.gz 2024-08-25T21:09:40.024 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56194.log 2024-08-25T21:09:40.025 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137805.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137805.log.gz 2024-08-25T21:09:40.025 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85559.log 2024-08-25T21:09:40.026 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56194.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56194.log.gz 2024-08-25T21:09:40.026 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71417.log 2024-08-25T21:09:40.027 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85559.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.85559.log.gz 2024-08-25T21:09:40.027 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.140801.log 2024-08-25T21:09:40.028 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71417.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71417.log.gz 2024-08-25T21:09:40.028 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70190.log 2024-08-25T21:09:40.029 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140801.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42441.log 2024-08-25T21:09:40.029 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140801.log.gz 2024-08-25T21:09:40.029 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70190.log.gz 2024-08-25T21:09:40.029 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83527.log 2024-08-25T21:09:40.030 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42441.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42441.log.gz 2024-08-25T21:09:40.030 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111258.log 2024-08-25T21:09:40.031 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83527.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83527.log.gz 2024-08-25T21:09:40.031 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63204.log 2024-08-25T21:09:40.031 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111258.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99297.log 2024-08-25T21:09:40.031 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111258.log.gz 2024-08-25T21:09:40.032 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63204.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63204.log.gz 2024-08-25T21:09:40.032 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24853.log 2024-08-25T21:09:40.033 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99297.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99297.log.gz 2024-08-25T21:09:40.033 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64868.log 2024-08-25T21:09:40.034 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24853.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24853.log.gz 2024-08-25T21:09:40.034 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136184.log 2024-08-25T21:09:40.034 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64868.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.64868.log.gz 2024-08-25T21:09:40.035 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118039.log 2024-08-25T21:09:40.035 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136184.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136184.log.gz 2024-08-25T21:09:40.035 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88920.log 2024-08-25T21:09:40.036 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.118039.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99810.log 2024-08-25T21:09:40.036 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118039.log.gz 2024-08-25T21:09:40.037 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88920.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88920.log.gz 2024-08-25T21:09:40.037 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112690.log 2024-08-25T21:09:40.037 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99810.log.gz 2024-08-25T21:09:40.038 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61737.log 2024-08-25T21:09:40.038 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112690.log.gz 2024-08-25T21:09:40.039 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93243.log 2024-08-25T21:09:40.040 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61737.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40759.log 2024-08-25T21:09:40.040 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61737.log.gz 2024-08-25T21:09:40.040 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93243.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93243.log.gz 2024-08-25T21:09:40.040 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82430.log 2024-08-25T21:09:40.041 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40759.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40759.log.gz 2024-08-25T21:09:40.041 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141093.log 2024-08-25T21:09:40.042 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82430.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82430.log.gz 2024-08-25T21:09:40.042 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74191.log 2024-08-25T21:09:40.042 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141093.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141093.log.gz 2024-08-25T21:09:40.043 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51421.log 2024-08-25T21:09:40.043 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74191.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.17471.log 2024-08-25T21:09:40.043 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74191.log.gz 2024-08-25T21:09:40.044 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51421.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51421.log.gz 2024-08-25T21:09:40.044 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140224.log 2024-08-25T21:09:40.045 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.17471.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.17471.log.gz 2024-08-25T21:09:40.045 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103406.log 2024-08-25T21:09:40.046 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.140224.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140224.log.gz 2024-08-25T21:09:40.046 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90140.log 2024-08-25T21:09:40.047 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103406.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103406.log.gz 2024-08-25T21:09:40.047 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22055.log 2024-08-25T21:09:40.048 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90140.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130709.log 2024-08-25T21:09:40.048 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90140.log.gz 2024-08-25T21:09:40.048 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22055.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22055.log.gz 2024-08-25T21:09:40.049 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36181.log 2024-08-25T21:09:40.049 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.130709.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79570.log 2024-08-25T21:09:40.050 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130709.log.gz 2024-08-25T21:09:40.050 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36181.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36181.log.gz 2024-08-25T21:09:40.050 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50164.log 2024-08-25T21:09:40.051 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79570.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79570.log.gz 2024-08-25T21:09:40.051 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25106.log 2024-08-25T21:09:40.051 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50164.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50164.log.gz 2024-08-25T21:09:40.052 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91016.log 2024-08-25T21:09:40.052 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25106.log.gz 2024-08-25T21:09:40.053 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103538.log 2024-08-25T21:09:40.053 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91016.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91016.log.gz 2024-08-25T21:09:40.054 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91014.log 2024-08-25T21:09:40.054 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103538.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120435.log 2024-08-25T21:09:40.054 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103538.log.gz 2024-08-25T21:09:40.055 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91014.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91014.log.gz 2024-08-25T21:09:40.055 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79658.log 2024-08-25T21:09:40.056 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120435.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.120435.log.gz -- 2024-08-25T21:09:40.056 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster2-client.admin.80546.log 2024-08-25T21:09:40.057 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79658.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79658.log.gz 2024-08-25T21:09:40.057 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122133.log 2024-08-25T21:09:40.057 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80546.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54048.log 2024-08-25T21:09:40.058 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80546.log.gz 2024-08-25T21:09:40.058 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122133.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122133.log.gz 2024-08-25T21:09:40.058 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134253.log 2024-08-25T21:09:40.059 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54048.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67472.log 2024-08-25T21:09:40.059 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54048.log.gz 2024-08-25T21:09:40.060 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134253.log.gz 2024-08-25T21:09:40.060 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111073.log 2024-08-25T21:09:40.061 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67472.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125845.log 2024-08-25T21:09:40.061 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67472.log.gz 2024-08-25T21:09:40.061 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111073.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111073.log.gz 2024-08-25T21:09:40.061 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51656.log 2024-08-25T21:09:40.062 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125845.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95959.log 2024-08-25T21:09:40.062 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125845.log.gz 2024-08-25T21:09:40.063 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51656.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51656.log.gz 2024-08-25T21:09:40.063 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29614.log 2024-08-25T21:09:40.064 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95959.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95959.log.gz 2024-08-25T21:09:40.065 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69591.log 2024-08-25T21:09:40.065 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29614.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29614.log.gz 2024-08-25T21:09:40.065 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120222.log 2024-08-25T21:09:40.065 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69591.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129191.log 2024-08-25T21:09:40.066 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69591.log.gz 2024-08-25T21:09:40.066 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120222.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120222.log.gz 2024-08-25T21:09:40.067 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49219.log 2024-08-25T21:09:40.073 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.129191.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94668.log 2024-08-25T21:09:40.073 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129191.log.gz 2024-08-25T21:09:40.073 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49219.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49219.log.gz 2024-08-25T21:09:40.073 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30872.log 2024-08-25T21:09:40.073 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94668.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141335.log 2024-08-25T21:09:40.074 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94668.log.gz 2024-08-25T21:09:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.30872.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30872.log.gz 2024-08-25T21:09:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29551.log 2024-08-25T21:09:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141335.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.141335.log.gz -5 2024-08-25T21:09:40.074 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.134551.log 2024-08-25T21:09:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29551.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29551.log.gz 2024-08-25T21:09:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105785.log 2024-08-25T21:09:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134551.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134551.log.gz 2024-08-25T21:09:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87513.log 2024-08-25T21:09:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.105785.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.105785.log.gz 2024-08-25T21:09:40.074 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109212.log 2024-08-25T21:09:40.075 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87513.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87513.log.gz 2024-08-25T21:09:40.076 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98088.log 2024-08-25T21:09:40.076 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109212.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109212.log.gz 2024-08-25T21:09:40.076 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130181.log 2024-08-25T21:09:40.076 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98088.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98088.log.gz 2024-08-25T21:09:40.076 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67692.log 2024-08-25T21:09:40.077 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.130181.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126943.log 2024-08-25T21:09:40.077 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130181.log.gz 2024-08-25T21:09:40.077 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67692.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67692.log.gz 2024-08-25T21:09:40.077 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36898.log 2024-08-25T21:09:40.078 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126943.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59538.log 2024-08-25T21:09:40.078 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126943.log.gz 2024-08-25T21:09:40.079 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36898.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36898.log.gz 2024-08-25T21:09:40.079 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116377.log 2024-08-25T21:09:40.079 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59538.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59538.log.gz 2024-08-25T21:09:40.080 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60458.log 2024-08-25T21:09:40.080 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116377.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116377.log.gz 2024-08-25T21:09:40.080 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89046.log 2024-08-25T21:09:40.081 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60458.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52245.log 2024-08-25T21:09:40.081 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60458.log.gz 2024-08-25T21:09:40.082 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89046.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89046.log.gz 2024-08-25T21:09:40.082 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41977.log 2024-08-25T21:09:40.083 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52245.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64614.log 2024-08-25T21:09:40.083 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52245.log.gz 2024-08-25T21:09:40.083 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41977.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41977.log.gz 2024-08-25T21:09:40.083 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140806.log 2024-08-25T21:09:40.084 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64614.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112334.log 2024-08-25T21:09:40.085 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64614.log.gz 2024-08-25T21:09:40.085 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140806.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140806.log.gz 2024-08-25T21:09:40.085 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61027.log 2024-08-25T21:09:40.086 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112334.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112334.log.gz 2024-08-25T21:09:40.086 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78098.log 2024-08-25T21:09:40.087 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61027.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61027.log.gz 2024-08-25T21:09:40.087 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121754.log 2024-08-25T21:09:40.088 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78098.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138723.log 2024-08-25T21:09:40.088 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78098.log.gz 2024-08-25T21:09:40.088 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121754.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121754.log.gz 2024-08-25T21:09:40.089 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32661.log 2024-08-25T21:09:40.089 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138723.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24615.log 2024-08-25T21:09:40.089 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138723.log.gz 2024-08-25T21:09:40.090 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32661.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32661.log.gz 2024-08-25T21:09:40.090 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91556.log 2024-08-25T21:09:40.091 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24615.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37359.log 2024-08-25T21:09:40.091 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24615.log.gz 2024-08-25T21:09:40.092 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91556.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91556.log.gz 2024-08-25T21:09:40.092 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141449.log 2024-08-25T21:09:40.092 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37359.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37359.log.gz 2024-08-25T21:09:40.092 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41263.log 2024-08-25T21:09:40.093 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141449.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141449.log.gz 2024-08-25T21:09:40.093 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121356.log 2024-08-25T21:09:40.094 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41263.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136061.log 2024-08-25T21:09:40.094 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41263.log.gz 2024-08-25T21:09:40.095 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121356.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121356.log.gz 2024-08-25T21:09:40.095 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52580.log 2024-08-25T21:09:40.096 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136061.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136061.log.gz 2024-08-25T21:09:40.096 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30392.log 2024-08-25T21:09:40.097 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52580.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52580.log.gz 2024-08-25T21:09:40.097 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44745.log 2024-08-25T21:09:40.097 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30392.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30392.log.gz 2024-08-25T21:09:40.097 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117193.log 2024-08-25T21:09:40.098 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44745.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88615.log 2024-08-25T21:09:40.098 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44745.log.gz 2024-08-25T21:09:40.099 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117193.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117193.log.gz 2024-08-25T21:09:40.099 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80927.log 2024-08-25T21:09:40.099 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88615.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.88615.log.gz -- 2024-08-25T21:09:40.100 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster2-client.admin.56772.log 2024-08-25T21:09:40.100 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80927.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80927.log.gz 2024-08-25T21:09:40.100 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109865.log 2024-08-25T21:09:40.101 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56772.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56772.log.gz 2024-08-25T21:09:40.102 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster2-client.mirror.115953.log 2024-08-25T21:09:40.102 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109865.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.109865.log.gz 2024-08-25T21:09:40.102 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135285.log 2024-08-25T21:09:40.103 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.115953.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23227.log 2024-08-25T21:09:40.103 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.115953.log.gz 2024-08-25T21:09:40.104 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135285.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135285.log.gz 2024-08-25T21:09:40.104 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68044.log 2024-08-25T21:09:40.105 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23227.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23227.log.gz 2024-08-25T21:09:40.105 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100064.log 2024-08-25T21:09:40.105 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68044.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68044.log.gz 2024-08-25T21:09:40.105 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90918.log 2024-08-25T21:09:40.106 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100064.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100064.log.gz 2024-08-25T21:09:40.106 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141020.log 2024-08-25T21:09:40.107 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90918.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88583.log 2024-08-25T21:09:40.107 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90918.log.gz 2024-08-25T21:09:40.108 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141020.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141020.log.gz 2024-08-25T21:09:40.108 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111075.log 2024-08-25T21:09:40.109 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88583.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88583.log.gz 2024-08-25T21:09:40.109 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40887.log 2024-08-25T21:09:40.110 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111075.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111075.log.gz 2024-08-25T21:09:40.110 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127970.log 2024-08-25T21:09:40.110 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40887.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40887.log.gz 2024-08-25T21:09:40.111 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39625.log 2024-08-25T21:09:40.111 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.127970.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123311.log 2024-08-25T21:09:40.111 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127970.log.gz 2024-08-25T21:09:40.112 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39625.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39625.log.gz 2024-08-25T21:09:40.112 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37152.log 2024-08-25T21:09:40.113 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123311.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44515.log 2024-08-25T21:09:40.113 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123311.log.gz 2024-08-25T21:09:40.114 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37152.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37152.log.gz 2024-08-25T21:09:40.114 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30452.log 2024-08-25T21:09:40.114 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47890.log 2024-08-25T21:09:40.115 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44515.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44515.log.gz 2024-08-25T21:09:40.115 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30452.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73770.log 2024-08-25T21:09:40.115 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30452.log.gz 2024-08-25T21:09:40.116 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47890.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47890.log.gz 2024-08-25T21:09:40.116 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135723.log 2024-08-25T21:09:40.117 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73770.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73770.log.gz 2024-08-25T21:09:40.118 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60965.log 2024-08-25T21:09:40.118 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135723.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135723.log.gz 2024-08-25T21:09:40.118 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109045.log 2024-08-25T21:09:40.119 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60965.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60965.log.gz 2024-08-25T21:09:40.119 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137112.log 2024-08-25T21:09:40.119 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109045.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109045.log.gz 2024-08-25T21:09:40.119 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136327.log 2024-08-25T21:09:40.120 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137112.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137112.log.gz 2024-08-25T21:09:40.121 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77827.log 2024-08-25T21:09:40.122 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136327.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136327.log.gz 2024-08-25T21:09:40.122 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26611.log 2024-08-25T21:09:40.122 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77827.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77827.log.gz 2024-08-25T21:09:40.122 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36118.log 2024-08-25T21:09:40.123 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26611.log.gz 2024-08-25T21:09:40.124 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.43207.log 2024-08-25T21:09:40.124 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36118.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36118.log.gz 2024-08-25T21:09:40.124 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30306.log 2024-08-25T21:09:40.125 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.43207.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.43207.log.gz 2024-08-25T21:09:40.125 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100796.log 2024-08-25T21:09:40.125 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30306.log.gz 2024-08-25T21:09:40.125 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67118.log 2024-08-25T21:09:40.127 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100796.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100796.log.gz 2024-08-25T21:09:40.127 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124235.log 2024-08-25T21:09:40.127 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67118.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67118.log.gz 2024-08-25T21:09:40.127 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70049.log 2024-08-25T21:09:40.128 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124235.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124235.log.gz 2024-08-25T21:09:40.128 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64448.log 2024-08-25T21:09:40.129 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70049.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70049.log.gz 2024-08-25T21:09:40.129 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101325.log 2024-08-25T21:09:40.129 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64448.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64448.log.gz 2024-08-25T21:09:40.129 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31676.log 2024-08-25T21:09:40.130 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101325.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101325.log.gz 2024-08-25T21:09:40.130 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70216.log 2024-08-25T21:09:40.131 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31676.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31676.log.gz 2024-08-25T21:09:40.131 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111217.log 2024-08-25T21:09:40.132 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70216.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70216.log.gz 2024-08-25T21:09:40.132 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133973.log 2024-08-25T21:09:40.132 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111217.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111217.log.gz 2024-08-25T21:09:40.132 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106428.log 2024-08-25T21:09:40.133 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133973.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.133973.log.gz 2024-08-25T21:09:40.133 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.50500.log 2024-08-25T21:09:40.133 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.106428.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106428.log.gz 2024-08-25T21:09:40.134 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140534.log 2024-08-25T21:09:40.134 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50500.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50500.log.gz 2024-08-25T21:09:40.135 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140585.log 2024-08-25T21:09:40.135 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140534.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140534.log.gz 2024-08-25T21:09:40.135 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124985.log 2024-08-25T21:09:40.136 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140585.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140585.log.gz 2024-08-25T21:09:40.136 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62284.log 2024-08-25T21:09:40.137 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124985.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124985.log.gz 2024-08-25T21:09:40.137 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72423.log 2024-08-25T21:09:40.138 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62284.log.gz 2024-08-25T21:09:40.138 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29109.log 2024-08-25T21:09:40.139 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72423.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72423.log.gz 2024-08-25T21:09:40.139 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117324.log 2024-08-25T21:09:40.139 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29109.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29109.log.gz 2024-08-25T21:09:40.139 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118127.log 2024-08-25T21:09:40.140 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117324.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117324.log.gz 2024-08-25T21:09:40.140 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103341.log 2024-08-25T21:09:40.141 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118127.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118127.log.gz 2024-08-25T21:09:40.141 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73558.log 2024-08-25T21:09:40.142 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103341.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103341.log.gz 2024-08-25T21:09:40.142 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36334.log 2024-08-25T21:09:40.143 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73558.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69604.log 2024-08-25T21:09:40.143 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73558.log.gz 2024-08-25T21:09:40.144 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36334.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36334.log.gz 2024-08-25T21:09:40.144 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62010.log 2024-08-25T21:09:40.145 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69604.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69604.log.gz 2024-08-25T21:09:40.145 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39376.log 2024-08-25T21:09:40.145 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62010.log.gz 2024-08-25T21:09:40.145 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60754.log 2024-08-25T21:09:40.146 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39376.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39376.log.gz 2024-08-25T21:09:40.146 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70672.log 2024-08-25T21:09:40.147 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60754.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60754.log.gz 2024-08-25T21:09:40.147 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83552.log 2024-08-25T21:09:40.147 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70672.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70672.log.gz 2024-08-25T21:09:40.147 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96410.log 2024-08-25T21:09:40.148 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83552.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83552.log.gz 2024-08-25T21:09:40.148 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139825.log 2024-08-25T21:09:40.149 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96410.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96410.log.gz 2024-08-25T21:09:40.149 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65958.log 2024-08-25T21:09:40.150 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.139825.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139825.log.gz 2024-08-25T21:09:40.150 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121605.log 2024-08-25T21:09:40.151 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65958.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65958.log.gz 2024-08-25T21:09:40.151 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52956.log 2024-08-25T21:09:40.152 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121605.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121605.log.gz 2024-08-25T21:09:40.152 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35565.log 2024-08-25T21:09:40.153 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52956.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52956.log.gz 2024-08-25T21:09:40.153 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.117624.log 2024-08-25T21:09:40.153 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35565.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35565.log.gz 2024-08-25T21:09:40.153 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132502.log 2024-08-25T21:09:40.154 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.117624.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.117624.log.gz 2024-08-25T21:09:40.154 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81387.log 2024-08-25T21:09:40.155 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132502.log.gz 2024-08-25T21:09:40.155 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71917.log 2024-08-25T21:09:40.156 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81387.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81387.log.gz 2024-08-25T21:09:40.156 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92014.log 2024-08-25T21:09:40.156 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.71917.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71917.log.gz 2024-08-25T21:09:40.156 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77868.log 2024-08-25T21:09:40.157 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92014.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92014.log.gz 2024-08-25T21:09:40.157 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129953.log 2024-08-25T21:09:40.158 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77868.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77868.log.gz 2024-08-25T21:09:40.158 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35528.log 2024-08-25T21:09:40.159 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129953.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129953.log.gz 2024-08-25T21:09:40.159 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106959.log 2024-08-25T21:09:40.160 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35528.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35528.log.gz 2024-08-25T21:09:40.160 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71659.log 2024-08-25T21:09:40.161 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106959.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106959.log.gz 2024-08-25T21:09:40.161 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139942.log 2024-08-25T21:09:40.161 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71659.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71659.log.gz 2024-08-25T21:09:40.161 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49177.log 2024-08-25T21:09:40.162 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139942.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139942.log.gz 2024-08-25T21:09:40.162 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54901.log 2024-08-25T21:09:40.163 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49177.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49177.log.gz 2024-08-25T21:09:40.163 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63901.log 2024-08-25T21:09:40.163 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54901.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54901.log.gz 2024-08-25T21:09:40.163 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61214.log 2024-08-25T21:09:40.164 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63901.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63901.log.gz 2024-08-25T21:09:40.164 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80594.log 2024-08-25T21:09:40.165 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61214.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61214.log.gz 2024-08-25T21:09:40.165 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91470.log 2024-08-25T21:09:40.166 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80594.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80594.log.gz 2024-08-25T21:09:40.166 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67822.log 2024-08-25T21:09:40.167 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91470.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91470.log.gz 2024-08-25T21:09:40.167 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85665.log 2024-08-25T21:09:40.167 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67822.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67822.log.gz 2024-08-25T21:09:40.167 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72220.log 2024-08-25T21:09:40.168 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85665.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85665.log.gz 2024-08-25T21:09:40.168 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79702.log 2024-08-25T21:09:40.169 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72220.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72220.log.gz 2024-08-25T21:09:40.169 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111280.log 2024-08-25T21:09:40.170 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79702.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80724.log 2024-08-25T21:09:40.170 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79702.log.gz 2024-08-25T21:09:40.170 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111280.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111280.log.gz 2024-08-25T21:09:40.171 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115540.log 2024-08-25T21:09:40.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80724.log: gzip 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80724.log.gz -5 --verbose 2024-08-25T21:09:40.171 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.96240.log 2024-08-25T21:09:40.172 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115540.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115540.log.gz 2024-08-25T21:09:40.172 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65959.log 2024-08-25T21:09:40.173 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96240.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66508.log 2024-08-25T21:09:40.173 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96240.log.gz 2024-08-25T21:09:40.174 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65959.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65959.log.gz 2024-08-25T21:09:40.174 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.36072.log 2024-08-25T21:09:40.175 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66508.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62095.log 2024-08-25T21:09:40.175 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66508.log.gz 2024-08-25T21:09:40.176 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.36072.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.36072.log.gz 2024-08-25T21:09:40.176 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73052.log 2024-08-25T21:09:40.176 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62095.log.gz 2024-08-25T21:09:40.177 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70906.log 2024-08-25T21:09:40.177 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73052.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73052.log.gz 2024-08-25T21:09:40.177 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96704.log 2024-08-25T21:09:40.178 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70906.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70906.log.gz 2024-08-25T21:09:40.178 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.93797.log 2024-08-25T21:09:40.179 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96704.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96704.log.gz 2024-08-25T21:09:40.179 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64282.log 2024-08-25T21:09:40.180 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.93797.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.93797.log.gz -5 2024-08-25T21:09:40.180 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.90582.log 2024-08-25T21:09:40.181 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64282.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64282.log.gz 2024-08-25T21:09:40.181 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97543.log 2024-08-25T21:09:40.181 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90582.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90582.log.gz 2024-08-25T21:09:40.182 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42504.log 2024-08-25T21:09:40.182 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97543.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97543.log.gz 2024-08-25T21:09:40.182 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112895.log 2024-08-25T21:09:40.183 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42504.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135222.log 2024-08-25T21:09:40.183 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42504.log.gz 2024-08-25T21:09:40.184 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112895.log.gz 2024-08-25T21:09:40.184 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126856.log 2024-08-25T21:09:40.185 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135222.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135222.log.gz 2024-08-25T21:09:40.185 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40676.log 2024-08-25T21:09:40.185 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126856.log.gz 2024-08-25T21:09:40.186 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73666.log 2024-08-25T21:09:40.187 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40676.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.33022.log 2024-08-25T21:09:40.187 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40676.log.gz 2024-08-25T21:09:40.187 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73666.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73666.log.gz 2024-08-25T21:09:40.187 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96213.log 2024-08-25T21:09:40.188 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.33022.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38134.log 2024-08-25T21:09:40.188 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.33022.log.gz 2024-08-25T21:09:40.189 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96213.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96213.log.gz 2024-08-25T21:09:40.189 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136394.log 2024-08-25T21:09:40.190 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38134.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55619.log 2024-08-25T21:09:40.190 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38134.log.gz 2024-08-25T21:09:40.191 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136394.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136394.log.gz 2024-08-25T21:09:40.191 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110694.log 2024-08-25T21:09:40.192 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55619.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55619.log.gz 2024-08-25T21:09:40.192 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90160.log 2024-08-25T21:09:40.192 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110694.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110694.log.gz 2024-08-25T21:09:40.192 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90656.log 2024-08-25T21:09:40.193 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90160.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.128175.log 2024-08-25T21:09:40.193 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90160.log.gz 2024-08-25T21:09:40.194 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90656.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90656.log.gz 2024-08-25T21:09:40.194 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120002.log 2024-08-25T21:09:40.195 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.128175.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58379.log 2024-08-25T21:09:40.195 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.128175.log.gz 2024-08-25T21:09:40.196 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120002.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120002.log.gz 2024-08-25T21:09:40.196 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115839.log 2024-08-25T21:09:40.196 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.58379.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58379.log.gz 2024-08-25T21:09:40.196 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.23027.log 2024-08-25T21:09:40.197 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115839.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115839.log.gz 2024-08-25T21:09:40.197 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109525.log 2024-08-25T21:09:40.198 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.23027.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67693.log 2024-08-25T21:09:40.198 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.23027.log.gz 2024-08-25T21:09:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109525.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109525.log.gz 2024-08-25T21:09:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78054.log 2024-08-25T21:09:40.200 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67693.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67693.log.gz 2024-08-25T21:09:40.200 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34209.log 2024-08-25T21:09:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78054.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78054.log.gz 2024-08-25T21:09:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58110.log 2024-08-25T21:09:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.34209.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99493.log 2024-08-25T21:09:40.202 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34209.log.gz 2024-08-25T21:09:40.202 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58110.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34371.log 2024-08-25T21:09:40.202 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58110.log.gz 2024-08-25T21:09:40.203 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99493.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99493.log.gz 2024-08-25T21:09:40.203 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88836.log 2024-08-25T21:09:40.204 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34371.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129230.log 0.0% 2024-08-25T21:09:40.204 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.34371.log.gz 2024-08-25T21:09:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88836.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88836.log.gz 2024-08-25T21:09:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67473.log 2024-08-25T21:09:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.129230.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129230.log.gzgzip 2024-08-25T21:09:40.206 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47188.log 2024-08-25T21:09:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67473.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141144.log 2024-08-25T21:09:40.207 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67473.log.gz 2024-08-25T21:09:40.207 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47188.log.gz 2024-08-25T21:09:40.207 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112477.log 2024-08-25T21:09:40.208 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141144.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141144.log.gz 2024-08-25T21:09:40.208 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106314.log 2024-08-25T21:09:40.209 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112477.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112477.log.gz 2024-08-25T21:09:40.209 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41433.log 2024-08-25T21:09:40.210 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.106314.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106314.log.gz 2024-08-25T21:09:40.210 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48305.log 2024-08-25T21:09:40.211 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41433.log.gz 2024-08-25T21:09:40.211 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25778.log 2024-08-25T21:09:40.212 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48305.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48305.log.gz 2024-08-25T21:09:40.212 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76690.log 2024-08-25T21:09:40.212 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25778.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25778.log.gz 2024-08-25T21:09:40.213 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35387.log 2024-08-25T21:09:40.214 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76690.log.gz 2024-08-25T21:09:40.214 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59778.log 2024-08-25T21:09:40.214 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35387.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35387.log.gz 2024-08-25T21:09:40.214 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54611.log 2024-08-25T21:09:40.215 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59778.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59778.log.gz 2024-08-25T21:09:40.215 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115974.log 2024-08-25T21:09:40.215 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54611.log.gz 2024-08-25T21:09:40.216 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117399.log 2024-08-25T21:09:40.216 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115974.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115974.log.gz 2024-08-25T21:09:40.216 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69488.log 2024-08-25T21:09:40.217 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117399.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster2-client.admin.117399.log.gz /var/log/ceph/cluster1-client.mirror.89529.log 2024-08-25T21:09:40.217 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:40.218 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69488.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69488.log.gz 2024-08-25T21:09:40.218 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37777.log 2024-08-25T21:09:40.219 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89529.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89529.log.gz 2024-08-25T21:09:40.219 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40131.log 2024-08-25T21:09:40.219 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37777.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37777.log.gz 2024-08-25T21:09:40.219 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127877.log 2024-08-25T21:09:40.220 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40131.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40131.log.gz 2024-08-25T21:09:40.220 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127026.log 2024-08-25T21:09:40.221 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.127877.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127877.log.gz 2024-08-25T21:09:40.221 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107737.log 2024-08-25T21:09:40.222 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127026.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127026.log.gz 2024-08-25T21:09:40.222 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38029.log 2024-08-25T21:09:40.222 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107737.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107737.log.gz 2024-08-25T21:09:40.222 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76228.log 2024-08-25T21:09:40.223 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38029.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38029.log.gz 2024-08-25T21:09:40.223 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124403.log 2024-08-25T21:09:40.224 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76228.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76228.log.gz 2024-08-25T21:09:40.224 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140182.log 2024-08-25T21:09:40.225 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124403.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124403.log.gz 2024-08-25T21:09:40.225 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99769.log 2024-08-25T21:09:40.225 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.140182.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.125472.log -- replaced with /var/log/ceph/cluster1-client.admin.140182.log.gz 2024-08-25T21:09:40.226 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:40.226 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99769.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99769.log.gz 2024-08-25T21:09:40.226 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34302.log 2024-08-25T21:09:40.227 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125472.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125472.log.gz 2024-08-25T21:09:40.227 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28141.log 2024-08-25T21:09:40.228 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34302.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34302.log.gz 2024-08-25T21:09:40.228 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107898.log 2024-08-25T21:09:40.229 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28141.log.gz 2024-08-25T21:09:40.229 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24070.log 2024-08-25T21:09:40.229 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107898.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140449.log 2024-08-25T21:09:40.230 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107898.log.gz 2024-08-25T21:09:40.230 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24070.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24070.log.gz 2024-08-25T21:09:40.230 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29654.log 2024-08-25T21:09:40.231 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.140449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140449.log.gz 2024-08-25T21:09:40.231 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122363.log 2024-08-25T21:09:40.232 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29654.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29654.log.gz 2024-08-25T21:09:40.232 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115320.log 2024-08-25T21:09:40.233 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122363.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122363.log.gz 2024-08-25T21:09:40.233 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34438.log 2024-08-25T21:09:40.233 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115320.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115320.log.gz 2024-08-25T21:09:40.234 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38827.log 2024-08-25T21:09:40.234 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.34438.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34438.log.gz 2024-08-25T21:09:40.234 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116056.log 2024-08-25T21:09:40.235 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38827.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38827.log.gz 2024-08-25T21:09:40.235 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83489.log 2024-08-25T21:09:40.236 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.116056.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.116056.log.gz 2024-08-25T21:09:40.236 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131539.log 2024-08-25T21:09:40.237 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83489.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83489.log.gz 2024-08-25T21:09:40.237 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115973.log 2024-08-25T21:09:40.237 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131539.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131539.log.gz 2024-08-25T21:09:40.238 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.142019.log 2024-08-25T21:09:40.238 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115973.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115973.log.gz 2024-08-25T21:09:40.239 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84826.log 2024-08-25T21:09:40.240 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.142019.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.142019.log.gz 2024-08-25T21:09:40.240 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68205.log 2024-08-25T21:09:40.240 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84826.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84826.log.gz 2024-08-25T21:09:40.240 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46199.log 2024-08-25T21:09:40.241 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68205.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68205.log.gz 2024-08-25T21:09:40.241 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64656.log 2024-08-25T21:09:40.242 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.46199.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46199.log.gz 2024-08-25T21:09:40.242 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78407.log 2024-08-25T21:09:40.242 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64656.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64656.log.gz 2024-08-25T21:09:40.242 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106863.log 2024-08-25T21:09:40.243 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78407.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38113.log 2024-08-25T21:09:40.244 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25801.log 2024-08-25T21:09:40.245 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25167.log 2024-08-25T21:09:40.245 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116609.log 2024-08-25T21:09:40.246 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29740.log 2024-08-25T21:09:40.247 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78407.log.gz 2024-08-25T21:09:40.247 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59537.log 2024-08-25T21:09:40.247 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106489.log 2024-08-25T21:09:40.248 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.116609.log: /var/log/ceph/cluster1-client.mirror.106863.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24223.log 2024-08-25T21:09:40.249 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116609.log.gz 2024-08-25T21:09:40.249 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50942.log 2024-08-25T21:09:40.251 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38113.log: /var/log/ceph/cluster1-client.mirror.25801.log: /var/log/ceph/cluster1-client.mirror.29740.log: 0.0%/var/log/ceph/cluster1-client.mirror.25167.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106863.log.gz -- replaced with /var/log/ceph/cluster1-client.mirror.38113.log.gz 2024-08-25T21:09:40.251 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:40.251 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24223.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25167.log.gz 2024-08-25T21:09:40.251 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25801.log.gz 2024-08-25T21:09:40.251 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29740.log.gz 2024-08-25T21:09:40.251 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24223.log.gz 2024-08-25T21:09:40.251 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59537.log: /var/log/ceph/cluster1-client.admin.106489.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81766.log 2024-08-25T21:09:40.251 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106489.log.gz 2024-08-25T21:09:40.251 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.59537.log.gz 2024-08-25T21:09:40.252 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50942.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50942.log.gz 2024-08-25T21:09:40.252 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.131881.log 2024-08-25T21:09:40.253 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.81766.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81766.log.gz 2024-08-25T21:09:40.253 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98425.log 2024-08-25T21:09:40.253 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.131881.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.131881.log.gz 2024-08-25T21:09:40.253 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26438.log 2024-08-25T21:09:40.254 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98425.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105604.log 0.0% 2024-08-25T21:09:40.255 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.98425.log.gz 2024-08-25T21:09:40.255 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26438.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.70076.log -- replaced with /var/log/ceph/cluster1-client.mirror.26438.log.gz 2024-08-25T21:09:40.255 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:40.256 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.105604.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105604.log.gz 2024-08-25T21:09:40.256 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81123.log 2024-08-25T21:09:40.257 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70076.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70076.log.gz 2024-08-25T21:09:40.257 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.130113.log 2024-08-25T21:09:40.257 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81123.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81123.log.gz 2024-08-25T21:09:40.257 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85204.log 2024-08-25T21:09:40.258 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.130113.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.130113.log.gz 2024-08-25T21:09:40.259 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96491.log 2024-08-25T21:09:40.259 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85204.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85204.log.gz 2024-08-25T21:09:40.259 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133680.log 2024-08-25T21:09:40.260 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96491.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96491.log.gz 2024-08-25T21:09:40.260 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24749.log 2024-08-25T21:09:40.260 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133680.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133680.log.gz 2024-08-25T21:09:40.260 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35527.log 2024-08-25T21:09:40.261 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24749.log.gz 2024-08-25T21:09:40.261 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58586.log 2024-08-25T21:09:40.262 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35527.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35527.log.gz 2024-08-25T21:09:40.262 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89404.log 2024-08-25T21:09:40.263 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58586.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58586.log.gz 2024-08-25T21:09:40.263 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63404.log 2024-08-25T21:09:40.263 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89404.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89404.log.gz 2024-08-25T21:09:40.263 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108713.log 2024-08-25T21:09:40.264 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63404.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112873.log 2024-08-25T21:09:40.264 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63404.log.gz 2024-08-25T21:09:40.265 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108713.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108713.log.gz 2024-08-25T21:09:40.265 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.72238.log 2024-08-25T21:09:40.266 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112873.log.gz 2024-08-25T21:09:40.266 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59141.log 2024-08-25T21:09:40.267 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.72238.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.72238.log.gz 2024-08-25T21:09:40.267 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53000.log 2024-08-25T21:09:40.268 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59141.log.gz 2024-08-25T21:09:40.268 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74274.log 2024-08-25T21:09:40.268 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53000.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53000.log.gz 2024-08-25T21:09:40.268 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.32940.log 2024-08-25T21:09:40.269 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74274.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74274.log.gz 2024-08-25T21:09:40.269 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29887.log 2024-08-25T21:09:40.269 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.32940.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32940.log.gz 2024-08-25T21:09:40.270 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114588.log 2024-08-25T21:09:40.270 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29887.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29887.log.gz 2024-08-25T21:09:40.270 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97793.log 2024-08-25T21:09:40.278 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114588.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114588.log.gz 2024-08-25T21:09:40.278 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39921.log 2024-08-25T21:09:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97793.log.gz 2024-08-25T21:09:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132814.log 2024-08-25T21:09:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39921.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39921.log.gz 2024-08-25T21:09:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.35101.log 2024-08-25T21:09:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132814.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132814.log.gz 2024-08-25T21:09:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131713.log 2024-08-25T21:09:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.35101.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.35101.log.gz 2024-08-25T21:09:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27911.log 2024-08-25T21:09:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131713.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131713.log.gz 2024-08-25T21:09:40.279 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103942.log 2024-08-25T21:09:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27911.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27911.log.gz 2024-08-25T21:09:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95101.log 2024-08-25T21:09:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103942.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103942.log.gz 2024-08-25T21:09:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31112.log 2024-08-25T21:09:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95101.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95101.log.gz 2024-08-25T21:09:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68626.log 2024-08-25T21:09:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31112.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31112.log.gz 2024-08-25T21:09:40.280 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.84044.log 2024-08-25T21:09:40.282 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68626.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68626.log.gz 2024-08-25T21:09:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103560.log 2024-08-25T21:09:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.84044.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.84044.log.gz 2024-08-25T21:09:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87555.log 2024-08-25T21:09:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103560.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103560.log.gz 2024-08-25T21:09:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126516.log 2024-08-25T21:09:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87555.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87555.log.gz 2024-08-25T21:09:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92022.log 2024-08-25T21:09:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126516.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126516.log.gz 2024-08-25T21:09:40.283 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28202.log 2024-08-25T21:09:40.285 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92022.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92022.log.gz 2024-08-25T21:09:40.285 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69190.log 2024-08-25T21:09:40.285 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28202.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28202.log.gz 2024-08-25T21:09:40.285 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55629.log 2024-08-25T21:09:40.285 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69190.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69190.log.gz 2024-08-25T21:09:40.285 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36854.log 2024-08-25T21:09:40.286 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55629.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55629.log.gz 2024-08-25T21:09:40.286 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53270.log 2024-08-25T21:09:40.286 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36854.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49282.log 2024-08-25T21:09:40.286 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36854.log.gz 2024-08-25T21:09:40.287 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53270.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53270.log.gz 2024-08-25T21:09:40.287 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128081.log 2024-08-25T21:09:40.288 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49282.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98594.log 0.0% 2024-08-25T21:09:40.288 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.49282.log.gz 2024-08-25T21:09:40.289 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.128081.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128081.log.gz 2024-08-25T21:09:40.290 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83949.log 2024-08-25T21:09:40.290 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98594.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98594.log.gz 2024-08-25T21:09:40.290 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93703.log 2024-08-25T21:09:40.290 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83949.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83949.log.gz 2024-08-25T21:09:40.291 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44305.log 2024-08-25T21:09:40.291 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.93703.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93703.log.gz 2024-08-25T21:09:40.291 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108668.log 2024-08-25T21:09:40.292 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44305.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44305.log.gz 2024-08-25T21:09:40.292 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50412.log 2024-08-25T21:09:40.293 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108668.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108668.log.gz 2024-08-25T21:09:40.293 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106888.log 2024-08-25T21:09:40.293 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50412.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50412.log.gz 2024-08-25T21:09:40.294 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44070.log 2024-08-25T21:09:40.294 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106888.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106888.log.gz 2024-08-25T21:09:40.294 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50542.log 2024-08-25T21:09:40.295 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44070.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44070.log.gz 2024-08-25T21:09:40.295 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70485.log 2024-08-25T21:09:40.296 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50542.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50542.log.gz 2024-08-25T21:09:40.296 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121954.log 2024-08-25T21:09:40.297 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70485.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70485.log.gz 2024-08-25T21:09:40.297 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23265.log 2024-08-25T21:09:40.297 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121954.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121954.log.gz 2024-08-25T21:09:40.298 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139554.log 2024-08-25T21:09:40.298 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23265.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23265.log.gz 2024-08-25T21:09:40.299 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24586.log 2024-08-25T21:09:40.299 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.139554.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139554.log.gz 2024-08-25T21:09:40.299 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110159.log 2024-08-25T21:09:40.300 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24586.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24586.log.gz 2024-08-25T21:09:40.300 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49262.log 2024-08-25T21:09:40.301 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110159.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46313.log 2024-08-25T21:09:40.301 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110159.log.gz 2024-08-25T21:09:40.301 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49262.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49262.log.gz 2024-08-25T21:09:40.301 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26061.log 2024-08-25T21:09:40.302 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.46313.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46313.log.gz 2024-08-25T21:09:40.303 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125260.log 2024-08-25T21:09:40.304 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26061.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26061.log.gz 2024-08-25T21:09:40.304 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71152.log 2024-08-25T21:09:40.304 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125260.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125260.log.gz 2024-08-25T21:09:40.304 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95235.log 2024-08-25T21:09:40.305 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71152.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71152.log.gz 2024-08-25T21:09:40.305 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43191.log 2024-08-25T21:09:40.306 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95235.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95235.log.gz 2024-08-25T21:09:40.306 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53671.log 2024-08-25T21:09:40.307 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43191.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43191.log.gz 2024-08-25T21:09:40.307 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28710.log 2024-08-25T21:09:40.307 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53671.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53671.log.gz 2024-08-25T21:09:40.307 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.22720.log 2024-08-25T21:09:40.308 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28710.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28710.log.gz 2024-08-25T21:09:40.309 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103296.log 2024-08-25T21:09:40.309 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.22720.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.22720.log.gz 2024-08-25T21:09:40.309 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24132.log 2024-08-25T21:09:40.310 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103296.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103296.log.gz 2024-08-25T21:09:40.310 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.22856.log 2024-08-25T21:09:40.311 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24132.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24132.log.gz 2024-08-25T21:09:40.311 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47427.log 2024-08-25T21:09:40.311 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.22856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.22856.log.gz 2024-08-25T21:09:40.311 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84407.log 2024-08-25T21:09:40.312 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47427.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47427.log.gz 2024-08-25T21:09:40.312 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137613.log 2024-08-25T21:09:40.313 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84407.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84407.log.gz 2024-08-25T21:09:40.313 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139409.log 2024-08-25T21:09:40.313 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137613.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137613.log.gz 2024-08-25T21:09:40.313 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48042.log 2024-08-25T21:09:40.314 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139409.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139409.log.gz 2024-08-25T21:09:40.314 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116917.log 2024-08-25T21:09:40.315 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48042.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48042.log.gz 2024-08-25T21:09:40.315 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.17264.log 2024-08-25T21:09:40.316 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.116917.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.116917.log.gz 2024-08-25T21:09:40.316 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52075.log 2024-08-25T21:09:40.316 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.17264.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.17264.log.gz 2024-08-25T21:09:40.317 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133802.log 2024-08-25T21:09:40.317 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52075.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52075.log.gz 2024-08-25T21:09:40.317 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125976.log 2024-08-25T21:09:40.318 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133802.log.gz 2024-08-25T21:09:40.318 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29695.log 2024-08-25T21:09:40.319 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125976.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125976.log.gz 2024-08-25T21:09:40.319 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26023.log 2024-08-25T21:09:40.320 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29695.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27513.log 2024-08-25T21:09:40.320 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29695.log.gz 2024-08-25T21:09:40.321 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26023.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26023.log.gz 2024-08-25T21:09:40.321 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79505.log 2024-08-25T21:09:40.321 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27513.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27513.log.gz 2024-08-25T21:09:40.321 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126269.log 2024-08-25T21:09:40.322 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79505.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79505.log.gz 2024-08-25T21:09:40.322 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33202.log 2024-08-25T21:09:40.323 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.126269.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126269.log.gz 2024-08-25T21:09:40.323 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71352.log 2024-08-25T21:09:40.323 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33202.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33202.log.gz 2024-08-25T21:09:40.323 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.23313.log 2024-08-25T21:09:40.324 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71352.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71352.log.gz 2024-08-25T21:09:40.324 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90722.log 2024-08-25T21:09:40.325 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.23313.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.23313.log.gz 2024-08-25T21:09:40.325 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82849.log 2024-08-25T21:09:40.327 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90722.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90722.log.gz 2024-08-25T21:09:40.327 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118842.log 2024-08-25T21:09:40.327 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82849.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82849.log.gz 2024-08-25T21:09:40.327 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110413.log 2024-08-25T21:09:40.328 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118842.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118842.log.gz 2024-08-25T21:09:40.328 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111512.log 2024-08-25T21:09:40.328 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110413.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110413.log.gz 2024-08-25T21:09:40.328 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75890.log 2024-08-25T21:09:40.330 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111512.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111512.log.gz 2024-08-25T21:09:40.330 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53461.log 2024-08-25T21:09:40.330 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75890.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75890.log.gz 2024-08-25T21:09:40.330 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142288.log 2024-08-25T21:09:40.331 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53461.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53461.log.gz 2024-08-25T21:09:40.331 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129751.log 2024-08-25T21:09:40.331 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.142288.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142288.log.gz 2024-08-25T21:09:40.331 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122783.log 2024-08-25T21:09:40.332 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129751.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129751.log.gz 2024-08-25T21:09:40.332 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24068.log 2024-08-25T21:09:40.333 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122783.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122783.log.gz 2024-08-25T21:09:40.333 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140583.log 2024-08-25T21:09:40.334 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24068.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24068.log.gz 2024-08-25T21:09:40.334 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32662.log 2024-08-25T21:09:40.335 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140583.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140583.log.gz 2024-08-25T21:09:40.335 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110972.log 2024-08-25T21:09:40.335 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32662.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32662.log.gz 2024-08-25T21:09:40.335 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41687.log 2024-08-25T21:09:40.337 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110972.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110972.log.gz 2024-08-25T21:09:40.337 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133369.log 2024-08-25T21:09:40.337 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41687.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41687.log.gz 2024-08-25T21:09:40.337 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110693.log 2024-08-25T21:09:40.338 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133369.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133369.log.gz 2024-08-25T21:09:40.338 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75553.log 2024-08-25T21:09:40.338 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110693.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110693.log.gz 2024-08-25T21:09:40.339 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112119.log 2024-08-25T21:09:40.339 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75553.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75553.log.gz 2024-08-25T21:09:40.339 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-osd.2.log 2024-08-25T21:09:40.340 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112119.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112119.log.gz 2024-08-25T21:09:40.340 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110695.log 2024-08-25T21:09:40.341 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-osd.2.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62348.log 2024-08-25T21:09:40.342 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110695.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110695.log.gz 2024-08-25T21:09:40.342 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132147.log 2024-08-25T21:09:40.342 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62348.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62348.log.gz 2024-08-25T21:09:40.343 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138024.log 2024-08-25T21:09:40.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132147.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132147.log.gz 2024-08-25T21:09:40.343 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81994.log 2024-08-25T21:09:40.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138024.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138024.log.gz 2024-08-25T21:09:40.344 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117603.log 2024-08-25T21:09:40.345 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.81994.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81994.log.gz 2024-08-25T21:09:40.345 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134805.log 2024-08-25T21:09:40.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.117603.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117603.log.gz 2024-08-25T21:09:40.347 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38953.log 2024-08-25T21:09:40.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134805.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134805.log.gz 2024-08-25T21:09:40.347 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46105.log 2024-08-25T21:09:40.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38953.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.57856.log 2024-08-25T21:09:40.348 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38953.log.gz 2024-08-25T21:09:40.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46105.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46105.log.gz 2024-08-25T21:09:40.348 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134803.log 2024-08-25T21:09:40.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.57856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.57856.log.gz 2024-08-25T21:09:40.349 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99598.log 2024-08-25T21:09:40.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134803.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134803.log.gz 2024-08-25T21:09:40.350 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62196.log 2024-08-25T21:09:40.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99598.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99598.log.gz 2024-08-25T21:09:40.351 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120038.log 2024-08-25T21:09:40.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62196.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62196.log.gz 2024-08-25T21:09:40.351 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128078.log 2024-08-25T21:09:40.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120038.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120038.log.gz 2024-08-25T21:09:40.353 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99811.log 2024-08-25T21:09:40.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.128078.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128078.log.gz 2024-08-25T21:09:40.353 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79438.log 2024-08-25T21:09:40.354 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99811.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99811.log.gz 2024-08-25T21:09:40.354 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41053.log 2024-08-25T21:09:40.355 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79438.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79438.log.gz 2024-08-25T21:09:40.355 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51782.log 2024-08-25T21:09:40.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41053.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41053.log.gz 2024-08-25T21:09:40.356 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78224.log 2024-08-25T21:09:40.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51782.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51782.log.gz 2024-08-25T21:09:40.356 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108774.log 2024-08-25T21:09:40.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78224.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78224.log.gz 2024-08-25T21:09:40.357 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81004.log 2024-08-25T21:09:40.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108774.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108774.log.gz 2024-08-25T21:09:40.358 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43742.log 2024-08-25T21:09:40.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81004.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81004.log.gz 2024-08-25T21:09:40.359 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41557.log 2024-08-25T21:09:40.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43742.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43742.log.gz 2024-08-25T21:09:40.359 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40551.log 2024-08-25T21:09:40.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41557.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41557.log.gz 2024-08-25T21:09:40.361 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66903.log 2024-08-25T21:09:40.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40551.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40551.log.gz 2024-08-25T21:09:40.361 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60710.log 2024-08-25T21:09:40.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66903.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66903.log.gz 2024-08-25T21:09:40.362 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101405.log 2024-08-25T21:09:40.363 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60710.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60710.log.gz 2024-08-25T21:09:40.363 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140173.log 2024-08-25T21:09:40.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101405.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101405.log.gz 2024-08-25T21:09:40.364 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34115.log 2024-08-25T21:09:40.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.140173.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140173.log.gz 2024-08-25T21:09:40.364 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116300.log 2024-08-25T21:09:40.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34115.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34115.log.gz 2024-08-25T21:09:40.366 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129204.log 2024-08-25T21:09:40.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.116300.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67846.log 2024-08-25T21:09:40.366 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116300.log.gz 2024-08-25T21:09:40.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.129204.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129204.log.gz 2024-08-25T21:09:40.367 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27449.log 2024-08-25T21:09:40.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67846.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67846.log.gz 2024-08-25T21:09:40.368 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36795.log 2024-08-25T21:09:40.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27449.log.gz 2024-08-25T21:09:40.368 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118536.log 2024-08-25T21:09:40.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36795.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36795.log.gz 2024-08-25T21:09:40.369 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22016.log 2024-08-25T21:09:40.370 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118536.log.gz 2024-08-25T21:09:40.370 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127578.log 2024-08-25T21:09:40.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22016.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22016.log.gz 2024-08-25T21:09:40.371 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101204.log 2024-08-25T21:09:40.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127578.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127578.log.gz 2024-08-25T21:09:40.371 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51948.log 2024-08-25T21:09:40.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101204.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101204.log.gz 2024-08-25T21:09:40.372 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34168.log 2024-08-25T21:09:40.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51948.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51948.log.gz 2024-08-25T21:09:40.373 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86316.log 2024-08-25T21:09:40.374 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34168.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34168.log.gz 2024-08-25T21:09:40.374 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51531.log 2024-08-25T21:09:40.374 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86316.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86316.log.gz 2024-08-25T21:09:40.375 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140952.log 2024-08-25T21:09:40.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51531.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51531.log.gz 2024-08-25T21:09:40.376 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37002.log 2024-08-25T21:09:40.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.140952.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140952.log.gz 2024-08-25T21:09:40.376 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70277.log 2024-08-25T21:09:40.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37002.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.76900.log -- replaced with /var/log/ceph/cluster1-client.mirror.37002.log.gz 2024-08-25T21:09:40.377 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:40.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70277.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70277.log.gz 2024-08-25T21:09:40.378 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43586.log 2024-08-25T21:09:40.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76900.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76900.log.gz 2024-08-25T21:09:40.378 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87828.log 2024-08-25T21:09:40.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43586.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43586.log.gz 2024-08-25T21:09:40.379 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120976.log 2024-08-25T21:09:40.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87828.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87828.log.gz 2024-08-25T21:09:40.381 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28150.log 2024-08-25T21:09:40.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120976.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120976.log.gz 2024-08-25T21:09:40.381 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34572.log 2024-08-25T21:09:40.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28150.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28150.log.gz 2024-08-25T21:09:40.382 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66745.log 2024-08-25T21:09:40.383 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34572.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34572.log.gz 2024-08-25T21:09:40.383 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122554.log 2024-08-25T21:09:40.383 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66745.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66745.log.gz 2024-08-25T21:09:40.383 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32663.log 2024-08-25T21:09:40.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122554.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122554.log.gz 2024-08-25T21:09:40.384 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122992.log 2024-08-25T21:09:40.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32663.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45092.log 2024-08-25T21:09:40.385 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32663.log.gz 2024-08-25T21:09:40.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122992.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122992.log.gz 2024-08-25T21:09:40.385 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67736.log 2024-08-25T21:09:40.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.45092.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45092.log.gz 2024-08-25T21:09:40.386 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82891.log 2024-08-25T21:09:40.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67736.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67736.log.gz 2024-08-25T21:09:40.387 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99748.log 2024-08-25T21:09:40.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82891.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82891.log.gz 2024-08-25T21:09:40.388 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71570.log 2024-08-25T21:09:40.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99748.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99748.log.gz 2024-08-25T21:09:40.389 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34458.log 2024-08-25T21:09:40.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71570.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71570.log.gz 2024-08-25T21:09:40.389 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114916.log 2024-08-25T21:09:40.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34458.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34458.log.gz 2024-08-25T21:09:40.390 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133719.log 2024-08-25T21:09:40.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114916.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.114916.log.gz --verbose 2024-08-25T21:09:40.391 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.admin.114266.log 2024-08-25T21:09:40.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133719.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133719.log.gz 2024-08-25T21:09:40.392 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.22425.log 2024-08-25T21:09:40.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114266.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114266.log.gz 2024-08-25T21:09:40.393 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135225.log 2024-08-25T21:09:40.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.22425.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.138147.log 2024-08-25T21:09:40.393 INFO:teuthology.orchestra.run.smithi121.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.22425.log.gz 2024-08-25T21:09:40.394 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135225.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135225.log.gz 2024-08-25T21:09:40.394 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40258.log 2024-08-25T21:09:40.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.138147.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.138147.log.gz 2024-08-25T21:09:40.395 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54218.log 2024-08-25T21:09:40.396 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40258.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40258.log.gz 2024-08-25T21:09:40.396 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97016.log 2024-08-25T21:09:40.396 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23999.log 2024-08-25T21:09:40.397 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54218.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54218.log.gz 2024-08-25T21:09:40.398 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97016.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97016.log.gz 2024-08-25T21:09:40.398 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121167.log 2024-08-25T21:09:40.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23999.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23999.log.gz 2024-08-25T21:09:40.399 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84531.log 2024-08-25T21:09:40.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121167.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121167.log.gz 2024-08-25T21:09:40.399 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120434.log 2024-08-25T21:09:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84531.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84531.log.gz 2024-08-25T21:09:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96828.log 2024-08-25T21:09:40.401 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120434.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120434.log.gz 2024-08-25T21:09:40.401 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74505.log 2024-08-25T21:09:40.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96828.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96828.log.gz 2024-08-25T21:09:40.402 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114479.log 2024-08-25T21:09:40.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74505.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74505.log.gz 2024-08-25T21:09:40.402 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55621.log 2024-08-25T21:09:40.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114479.log.gz 2024-08-25T21:09:40.403 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55313.log 2024-08-25T21:09:40.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55621.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55621.log.gz 2024-08-25T21:09:40.404 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66680.log 2024-08-25T21:09:40.405 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55313.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55313.log.gz 2024-08-25T21:09:40.405 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93662.log 2024-08-25T21:09:40.405 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66680.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.admin.66680.log.gz 2024-08-25T21:09:40.405 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.76651.log 2024-08-25T21:09:40.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.93662.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93662.log.gz 2024-08-25T21:09:40.406 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51571.log 2024-08-25T21:09:40.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76651.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76651.log.gz 2024-08-25T21:09:40.408 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33533.log 2024-08-25T21:09:40.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51571.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51571.log.gz 2024-08-25T21:09:40.408 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71088.log 2024-08-25T21:09:40.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33533.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33533.log.gz 2024-08-25T21:09:40.409 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87322.log 2024-08-25T21:09:40.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71088.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71088.log.gz 2024-08-25T21:09:40.410 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106826.log 2024-08-25T21:09:40.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87322.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87322.log.gz 2024-08-25T21:09:40.411 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100681.log 2024-08-25T21:09:40.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106826.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106826.log.gz 2024-08-25T21:09:40.411 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28291.log 2024-08-25T21:09:40.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100681.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100681.log.gz 2024-08-25T21:09:40.412 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113885.log 2024-08-25T21:09:40.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28291.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28291.log.gz 2024-08-25T21:09:40.413 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70236.log 2024-08-25T21:09:40.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113885.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113885.log.gz 2024-08-25T21:09:40.414 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99289.log 2024-08-25T21:09:40.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70236.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70236.log.gz 2024-08-25T21:09:40.414 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124463.log 2024-08-25T21:09:40.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99289.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99289.log.gz 2024-08-25T21:09:40.415 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26621.log 2024-08-25T21:09:40.416 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124463.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124463.log.gz 2024-08-25T21:09:40.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30202.log 2024-08-25T21:09:40.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26621.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26621.log.gz 2024-08-25T21:09:40.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133850.log 2024-08-25T21:09:40.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30202.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30202.log.gz 2024-08-25T21:09:40.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113505.log 2024-08-25T21:09:40.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133850.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133850.log.gz 2024-08-25T21:09:40.418 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41347.log 2024-08-25T21:09:40.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113505.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113505.log.gz 2024-08-25T21:09:40.419 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96493.log 2024-08-25T21:09:40.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41347.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41347.log.gz 2024-08-25T21:09:40.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108504.log 2024-08-25T21:09:40.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96493.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96493.log.gz 2024-08-25T21:09:40.421 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70827.log 2024-08-25T21:09:40.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108504.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108504.log.gz 2024-08-25T21:09:40.422 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129383.log 2024-08-25T21:09:40.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70827.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70827.log.gz 2024-08-25T21:09:40.423 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62032.log 2024-08-25T21:09:40.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129383.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129383.log.gz 2024-08-25T21:09:40.424 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34800.log 2024-08-25T21:09:40.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62032.log.gz 2024-08-25T21:09:40.424 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49471.log 2024-08-25T21:09:40.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34800.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34800.log.gz 2024-08-25T21:09:40.425 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91182.log 2024-08-25T21:09:40.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49471.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49471.log.gz 2024-08-25T21:09:40.426 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133300.log 2024-08-25T21:09:40.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91182.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91182.log.gz 2024-08-25T21:09:40.427 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81560.log 2024-08-25T21:09:40.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133300.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133300.log.gz 2024-08-25T21:09:40.428 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69707.log 2024-08-25T21:09:40.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81560.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81560.log.gz 2024-08-25T21:09:40.429 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116781.log 2024-08-25T21:09:40.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.69707.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69707.log.gz 2024-08-25T21:09:40.430 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121019.log 2024-08-25T21:09:40.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116781.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116781.log.gz 2024-08-25T21:09:40.431 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25377.log 2024-08-25T21:09:40.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121019.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95423.log 2024-08-25T21:09:40.432 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121019.log.gz 2024-08-25T21:09:40.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25377.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25377.log.gz 2024-08-25T21:09:40.433 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117785.log 2024-08-25T21:09:40.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95423.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95423.log.gz 2024-08-25T21:09:40.433 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132690.log 2024-08-25T21:09:40.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.117785.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34759.log 2024-08-25T21:09:40.434 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117785.log.gz 2024-08-25T21:09:40.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132690.log.gz 2024-08-25T21:09:40.435 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34967.log 2024-08-25T21:09:40.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34759.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34759.log.gz 2024-08-25T21:09:40.436 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25063.log 2024-08-25T21:09:40.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34967.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.34967.log.gz -5 2024-08-25T21:09:40.436 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.110605.log 2024-08-25T21:09:40.437 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25063.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25063.log.gz 2024-08-25T21:09:40.437 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72380.log 2024-08-25T21:09:40.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110605.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110605.log.gz 2024-08-25T21:09:40.438 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.72052.log 2024-08-25T21:09:40.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72380.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72380.log.gz 2024-08-25T21:09:40.439 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112971.log 2024-08-25T21:09:40.440 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.72052.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.72052.log.gz 2024-08-25T21:09:40.440 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138265.log 2024-08-25T21:09:40.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112971.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112971.log.gz 2024-08-25T21:09:40.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.31050.log 2024-08-25T21:09:40.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138265.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138265.log.gz 2024-08-25T21:09:40.442 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29199.log 2024-08-25T21:09:40.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.31050.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.31050.log.gz 2024-08-25T21:09:40.443 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134443.log 2024-08-25T21:09:40.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29199.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29199.log.gz 2024-08-25T21:09:40.444 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39709.log 2024-08-25T21:09:40.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134443.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134443.log.gz 2024-08-25T21:09:40.444 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26021.log 2024-08-25T21:09:40.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39709.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39709.log.gz 2024-08-25T21:09:40.445 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95387.log 2024-08-25T21:09:40.446 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109950.log 2024-08-25T21:09:40.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26021.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26021.log.gz 2024-08-25T21:09:40.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95387.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80633.log 2024-08-25T21:09:40.447 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95387.log.gz 2024-08-25T21:09:40.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109950.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109950.log.gz 2024-08-25T21:09:40.448 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28720.log 2024-08-25T21:09:40.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80633.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80633.log.gz 2024-08-25T21:09:40.449 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62491.log 2024-08-25T21:09:40.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28720.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28720.log.gz 2024-08-25T21:09:40.449 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93756.log 2024-08-25T21:09:40.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62491.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119212.log 2024-08-25T21:09:40.450 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62491.log.gz 2024-08-25T21:09:40.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.93756.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93756.log.gz 2024-08-25T21:09:40.451 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116755.log 2024-08-25T21:09:40.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119212.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115672.log 2024-08-25T21:09:40.452 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119212.log.gz 2024-08-25T21:09:40.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116755.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116755.log.gz 2024-08-25T21:09:40.453 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26948.log 2024-08-25T21:09:40.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115672.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115672.log.gz 2024-08-25T21:09:40.454 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84994.log 2024-08-25T21:09:40.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26948.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26948.log.gz 2024-08-25T21:09:40.455 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30161.log 2024-08-25T21:09:40.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84994.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84994.log.gz 2024-08-25T21:09:40.456 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124062.log 2024-08-25T21:09:40.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30161.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30161.log.gz 2024-08-25T21:09:40.456 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109717.log 2024-08-25T21:09:40.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124062.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94837.log 2024-08-25T21:09:40.457 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124062.log.gz 2024-08-25T21:09:40.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109717.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109717.log.gz 2024-08-25T21:09:40.458 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103094.log 2024-08-25T21:09:40.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94837.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94837.log.gz 2024-08-25T21:09:40.459 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132232.log 2024-08-25T21:09:40.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103094.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103094.log.gz 2024-08-25T21:09:40.460 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41616.log 2024-08-25T21:09:40.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132232.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132232.log.gz 2024-08-25T21:09:40.460 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118100.log 2024-08-25T21:09:40.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41616.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41616.log.gz 2024-08-25T21:09:40.461 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40257.log 2024-08-25T21:09:40.462 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118100.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50943.log 0.0% 2024-08-25T21:09:40.462 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.118100.log.gz 2024-08-25T21:09:40.463 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40257.log.gz 2024-08-25T21:09:40.463 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55730.log 2024-08-25T21:09:40.463 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50943.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50943.log.gz 2024-08-25T21:09:40.464 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-osd.0.log 2024-08-25T21:09:40.464 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55730.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55730.log.gz 2024-08-25T21:09:40.464 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43190.log 2024-08-25T21:09:40.465 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-osd.0.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.22771.log 2024-08-25T21:09:40.466 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43190.log.gz 2024-08-25T21:09:40.466 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71390.log 2024-08-25T21:09:40.467 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.22771.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.22771.log.gz 2024-08-25T21:09:40.467 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113358.log 2024-08-25T21:09:40.468 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71390.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71390.log.gz 2024-08-25T21:09:40.468 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97142.log 2024-08-25T21:09:40.468 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113358.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113358.log.gz 2024-08-25T21:09:40.468 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67890.log 2024-08-25T21:09:40.469 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97142.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97142.log.gz 2024-08-25T21:09:40.470 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70304.log 2024-08-25T21:09:40.470 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67890.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67890.log.gz 2024-08-25T21:09:40.470 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72843.log 2024-08-25T21:09:40.471 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70304.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70304.log.gz 2024-08-25T21:09:40.471 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52329.log 2024-08-25T21:09:40.471 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72843.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72843.log.gz 2024-08-25T21:09:40.471 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107480.log 2024-08-25T21:09:40.472 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52329.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52329.log.gz 2024-08-25T21:09:40.473 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133549.log 2024-08-25T21:09:40.473 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.107480.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107480.log.gz 2024-08-25T21:09:40.473 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59343.log 2024-08-25T21:09:40.474 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133549.log.gz 2024-08-25T21:09:40.474 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122446.log 2024-08-25T21:09:40.482 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59343.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59343.log.gz 2024-08-25T21:09:40.482 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53104.log 2024-08-25T21:09:40.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122446.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122446.log.gz 2024-08-25T21:09:40.483 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77720.log 2024-08-25T21:09:40.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53104.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53104.log.gz 2024-08-25T21:09:40.483 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105878.log 2024-08-25T21:09:40.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77720.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77720.log.gz 2024-08-25T21:09:40.483 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27366.log 2024-08-25T21:09:40.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105878.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80827.log 2024-08-25T21:09:40.483 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105878.log.gz 2024-08-25T21:09:40.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27366.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27366.log.gz 2024-08-25T21:09:40.483 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23624.log 2024-08-25T21:09:40.484 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80827.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80827.log.gz 2024-08-25T21:09:40.484 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104791.log 2024-08-25T21:09:40.484 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23624.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23624.log.gz 2024-08-25T21:09:40.484 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37610.log 2024-08-25T21:09:40.484 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.104791.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104791.log.gz 2024-08-25T21:09:40.484 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132021.log 2024-08-25T21:09:40.484 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37610.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37610.log.gz 2024-08-25T21:09:40.484 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62264.log 2024-08-25T21:09:40.484 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132021.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132021.log.gzgzip 2024-08-25T21:09:40.484 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65135.log 2024-08-25T21:09:40.487 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62264.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62264.log.gz 2024-08-25T21:09:40.487 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128229.log 2024-08-25T21:09:40.487 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65515.log 2024-08-25T21:09:40.487 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65135.log.gz 2024-08-25T21:09:40.487 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128229.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128229.log.gz 2024-08-25T21:09:40.487 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108005.log 2024-08-25T21:09:40.487 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65515.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65515.log.gz 2024-08-25T21:09:40.487 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31741.log 2024-08-25T21:09:40.487 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108005.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108005.log.gz 2024-08-25T21:09:40.487 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140720.log 2024-08-25T21:09:40.489 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31741.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.31741.log.gz 2024-08-25T21:09:40.489 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.84574.log 2024-08-25T21:09:40.489 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140720.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140720.log.gz 2024-08-25T21:09:40.489 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89213.log 2024-08-25T21:09:40.489 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84574.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84574.log.gz 2024-08-25T21:09:40.489 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71415.log 2024-08-25T21:09:40.490 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89213.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89213.log.gz 2024-08-25T21:09:40.490 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38871.log 2024-08-25T21:09:40.490 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71415.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71415.log.gz 2024-08-25T21:09:40.491 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124047.log 2024-08-25T21:09:40.491 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38871.log.gz 2024-08-25T21:09:40.491 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110963.log 2024-08-25T21:09:40.492 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124047.log.gzgzip 2024-08-25T21:09:40.492 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89824.log 2024-08-25T21:09:40.493 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110963.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110963.log.gz 2024-08-25T21:09:40.493 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97499.log 2024-08-25T21:09:40.494 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89824.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89824.log.gz 2024-08-25T21:09:40.494 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92076.log 2024-08-25T21:09:40.494 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97499.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97499.log.gz 2024-08-25T21:09:40.495 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29464.log 2024-08-25T21:09:40.495 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92076.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92076.log.gz 2024-08-25T21:09:40.495 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129839.log 2024-08-25T21:09:40.496 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29464.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29464.log.gz 2024-08-25T21:09:40.496 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117135.log 2024-08-25T21:09:40.497 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129839.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129839.log.gz 2024-08-25T21:09:40.497 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.31007.log 2024-08-25T21:09:40.497 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117135.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117135.log.gz 2024-08-25T21:09:40.498 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135576.log 2024-08-25T21:09:40.498 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.31007.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.31007.log.gz 2024-08-25T21:09:40.498 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63984.log 2024-08-25T21:09:40.499 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135576.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135576.log.gz 2024-08-25T21:09:40.499 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47259.log 2024-08-25T21:09:40.500 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63984.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63984.log.gz 2024-08-25T21:09:40.500 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86632.log 2024-08-25T21:09:40.500 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47259.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47259.log.gz 2024-08-25T21:09:40.501 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56109.log 2024-08-25T21:09:40.501 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86632.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86632.log.gz 2024-08-25T21:09:40.501 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65790.log 2024-08-25T21:09:40.502 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56109.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56109.log.gzgzip 2024-08-25T21:09:40.502 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128394.log 2024-08-25T21:09:40.503 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65790.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65790.log.gz 2024-08-25T21:09:40.503 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87640.log 2024-08-25T21:09:40.504 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.128394.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128394.log.gz 2024-08-25T21:09:40.504 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74774.log 2024-08-25T21:09:40.505 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87640.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87640.log.gz 2024-08-25T21:09:40.505 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53712.log 2024-08-25T21:09:40.506 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74774.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74774.log.gz 2024-08-25T21:09:40.506 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119517.log 2024-08-25T21:09:40.506 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53712.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53712.log.gz 2024-08-25T21:09:40.506 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69617.log 2024-08-25T21:09:40.507 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119517.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119517.log.gz 2024-08-25T21:09:40.507 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139187.log 2024-08-25T21:09:40.507 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69617.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69617.log.gz 2024-08-25T21:09:40.507 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75640.log 2024-08-25T21:09:40.508 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139187.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139187.log.gz 2024-08-25T21:09:40.508 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141156.log 2024-08-25T21:09:40.509 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75640.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75640.log.gz 2024-08-25T21:09:40.509 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92807.log 2024-08-25T21:09:40.510 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141156.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141156.log.gz 2024-08-25T21:09:40.510 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45345.log 2024-08-25T21:09:40.510 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92807.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92807.log.gz 2024-08-25T21:09:40.510 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92236.log 2024-08-25T21:09:40.511 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.45345.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45345.log.gz 2024-08-25T21:09:40.512 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103231.log 2024-08-25T21:09:40.512 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.92236.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.92236.log.gz 2024-08-25T21:09:40.512 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98046.log 2024-08-25T21:09:40.513 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103231.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103231.log.gz 2024-08-25T21:09:40.513 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119042.log 2024-08-25T21:09:40.514 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98046.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92590.log 2024-08-25T21:09:40.514 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98046.log.gz 2024-08-25T21:09:40.515 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119042.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119042.log.gz 2024-08-25T21:09:40.515 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89684.log 2024-08-25T21:09:40.515 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92590.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92590.log.gz 2024-08-25T21:09:40.515 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110346.log 2024-08-25T21:09:40.516 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89684.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89684.log.gz 2024-08-25T21:09:40.516 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91644.log 2024-08-25T21:09:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110346.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110346.log.gz 2024-08-25T21:09:40.517 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69099.log 2024-08-25T21:09:40.518 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91644.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.admin.91644.log.gz -- 2024-08-25T21:09:40.518 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.50732.log 2024-08-25T21:09:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69099.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69099.log.gz 2024-08-25T21:09:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109505.log 2024-08-25T21:09:40.519 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50732.log.gz 2024-08-25T21:09:40.520 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109464.log 2024-08-25T21:09:40.520 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109505.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139527.log 2024-08-25T21:09:40.520 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109505.log.gz 2024-08-25T21:09:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109464.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109464.log.gz 2024-08-25T21:09:40.521 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.32944.log 2024-08-25T21:09:40.522 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.139527.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122045.log 2024-08-25T21:09:40.522 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139527.log.gz 2024-08-25T21:09:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.32944.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32944.log.gz 2024-08-25T21:09:40.523 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124044.log 2024-08-25T21:09:40.524 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122045.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122045.log.gz 2024-08-25T21:09:40.524 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67967.log 2024-08-25T21:09:40.524 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124044.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124044.log.gz 2024-08-25T21:09:40.524 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71506.log 2024-08-25T21:09:40.525 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67967.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67967.log.gz 2024-08-25T21:09:40.525 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111569.log 2024-08-25T21:09:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71506.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71506.log.gz 2024-08-25T21:09:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49076.log 2024-08-25T21:09:40.527 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111569.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.108291.log 2024-08-25T21:09:40.527 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.111569.log.gz 2024-08-25T21:09:40.528 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49076.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49076.log.gz 2024-08-25T21:09:40.528 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130418.log 2024-08-25T21:09:40.528 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108291.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108291.log.gz 2024-08-25T21:09:40.528 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85623.log 2024-08-25T21:09:40.529 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130418.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111977.log 2024-08-25T21:09:40.529 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130418.log.gz 2024-08-25T21:09:40.530 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85623.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85623.log.gz 2024-08-25T21:09:40.530 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37064.log 2024-08-25T21:09:40.531 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111977.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111977.log.gz 2024-08-25T21:09:40.531 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.21928.log 2024-08-25T21:09:40.532 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37064.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37064.log.gz 2024-08-25T21:09:40.532 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87409.log 2024-08-25T21:09:40.533 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.21928.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.21928.log.gz 2024-08-25T21:09:40.533 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29781.log 2024-08-25T21:09:40.534 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87409.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87409.log.gz 2024-08-25T21:09:40.534 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80831.log 2024-08-25T21:09:40.534 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29781.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29781.log.gz 2024-08-25T21:09:40.534 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27262.log 2024-08-25T21:09:40.535 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.80831.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80831.log.gz 2024-08-25T21:09:40.535 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85980.log 2024-08-25T21:09:40.536 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27262.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114096.log 2024-08-25T21:09:40.536 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27262.log.gz 2024-08-25T21:09:40.537 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85980.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85980.log.gz 2024-08-25T21:09:40.537 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113622.log 2024-08-25T21:09:40.538 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114096.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114096.log.gz 2024-08-25T21:09:40.538 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54913.log 2024-08-25T21:09:40.538 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113622.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66128.log 2024-08-25T21:09:40.538 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113622.log.gz 2024-08-25T21:09:40.539 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54913.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54913.log.gz 2024-08-25T21:09:40.539 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117644.log 2024-08-25T21:09:40.540 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66128.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128310.log 2024-08-25T21:09:40.540 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66128.log.gz 2024-08-25T21:09:40.541 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.117644.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117644.log.gz 2024-08-25T21:09:40.541 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130889.log 2024-08-25T21:09:40.541 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128310.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126406.log 2024-08-25T21:09:40.542 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128310.log.gz 2024-08-25T21:09:40.543 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130889.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139077.log 2024-08-25T21:09:40.543 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130889.log.gz 2024-08-25T21:09:40.543 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.126406.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126406.log.gz 2024-08-25T21:09:40.543 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129637.log 2024-08-25T21:09:40.544 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139077.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139077.log.gz 2024-08-25T21:09:40.544 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103181.log 2024-08-25T21:09:40.545 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129637.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129637.log.gz 2024-08-25T21:09:40.545 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62136.log 2024-08-25T21:09:40.546 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103181.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103181.log.gz 2024-08-25T21:09:40.546 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137051.log 2024-08-25T21:09:40.547 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62136.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62136.log.gz 2024-08-25T21:09:40.547 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52707.log 2024-08-25T21:09:40.547 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137051.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84021.log 2024-08-25T21:09:40.547 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137051.log.gz 2024-08-25T21:09:40.548 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52707.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52707.log.gz 2024-08-25T21:09:40.548 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135433.log 2024-08-25T21:09:40.549 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84021.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99435.log 2024-08-25T21:09:40.549 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84021.log.gz 2024-08-25T21:09:40.550 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135433.log.gz 2024-08-25T21:09:40.550 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25841.log 2024-08-25T21:09:40.551 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99435.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105691.log 2024-08-25T21:09:40.551 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99435.log.gz 2024-08-25T21:09:40.551 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25841.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25841.log.gz 2024-08-25T21:09:40.551 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107527.log 2024-08-25T21:09:40.552 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.105691.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105691.log.gz 2024-08-25T21:09:40.552 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69191.log 2024-08-25T21:09:40.553 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107527.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77363.log 2024-08-25T21:09:40.553 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107527.log.gz 2024-08-25T21:09:40.554 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69191.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69191.log.gz 2024-08-25T21:09:40.554 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28856.log 2024-08-25T21:09:40.554 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77363.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82450.log 2024-08-25T21:09:40.554 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77363.log.gz 2024-08-25T21:09:40.555 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28856.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.28856.log.gz --verbose 2024-08-25T21:09:40.555 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.72631.log 2024-08-25T21:09:40.556 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82450.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82450.log.gz 2024-08-25T21:09:40.556 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45505.log 2024-08-25T21:09:40.557 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72631.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115800.log 2024-08-25T21:09:40.557 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72631.log.gz 2024-08-25T21:09:40.557 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45505.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45505.log.gz 2024-08-25T21:09:40.558 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104989.log 2024-08-25T21:09:40.558 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115800.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.admin.115800.log.gz 2024-08-25T21:09:40.558 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.admin.35339.log 2024-08-25T21:09:40.559 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104989.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104989.log.gz 2024-08-25T21:09:40.559 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110474.log 2024-08-25T21:09:40.560 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.35339.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136651.log 2024-08-25T21:09:40.560 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35339.log.gz 2024-08-25T21:09:40.561 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110474.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.mirror.110474.log.gz 2024-08-25T21:09:40.561 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.96345.log 2024-08-25T21:09:40.562 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136651.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136651.log.gz 2024-08-25T21:09:40.562 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70163.log 2024-08-25T21:09:40.562 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96345.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64824.log 2024-08-25T21:09:40.562 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96345.log.gz 2024-08-25T21:09:40.563 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70163.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70163.log.gz 2024-08-25T21:09:40.563 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61172.log 2024-08-25T21:09:40.564 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64824.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132773.log 2024-08-25T21:09:40.564 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64824.log.gz 2024-08-25T21:09:40.565 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61172.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61172.log.gz 2024-08-25T21:09:40.565 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40889.log 2024-08-25T21:09:40.565 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132773.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50864.log 2024-08-25T21:09:40.565 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132773.log.gz 2024-08-25T21:09:40.566 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40889.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.40889.log.gz --verbose 2024-08-25T21:09:40.566 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.75174.log 2024-08-25T21:09:40.567 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50864.log.gz 2024-08-25T21:09:40.567 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86187.log 2024-08-25T21:09:40.568 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75174.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54050.log 2024-08-25T21:09:40.568 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75174.log.gz 2024-08-25T21:09:40.569 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86187.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86187.log.gz 2024-08-25T21:09:40.569 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70578.log 2024-08-25T21:09:40.570 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54050.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37883.log 2024-08-25T21:09:40.570 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54050.log.gz 2024-08-25T21:09:40.570 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70578.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70578.log.gz 2024-08-25T21:09:40.570 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62430.log 2024-08-25T21:09:40.571 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37883.log.gz 2024-08-25T21:09:40.571 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85455.log 2024-08-25T21:09:40.572 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62430.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62430.log.gz -5 2024-08-25T21:09:40.572 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.112774.log 2024-08-25T21:09:40.573 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85455.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85455.log.gz 2024-08-25T21:09:40.573 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113276.log 2024-08-25T21:09:40.573 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112774.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112774.log.gz 2024-08-25T21:09:40.574 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31329.log 2024-08-25T21:09:40.574 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113276.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113276.log.gz 2024-08-25T21:09:40.574 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67847.log 2024-08-25T21:09:40.575 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31329.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88120.log 2024-08-25T21:09:40.575 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31329.log.gz 2024-08-25T21:09:40.576 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67847.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67847.log.gz 2024-08-25T21:09:40.576 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61590.log 2024-08-25T21:09:40.577 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88120.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87974.log 2024-08-25T21:09:40.577 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88120.log.gz 2024-08-25T21:09:40.577 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61590.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61590.log.gz 2024-08-25T21:09:40.577 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46675.log 2024-08-25T21:09:40.578 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87974.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124875.log 2024-08-25T21:09:40.578 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87974.log.gz 2024-08-25T21:09:40.579 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46675.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46675.log.gz 2024-08-25T21:09:40.579 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128718.log 2024-08-25T21:09:40.580 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124875.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124875.log.gz 2024-08-25T21:09:40.580 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93472.log 2024-08-25T21:09:40.581 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128718.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128718.log.gz 2024-08-25T21:09:40.581 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139408.log 2024-08-25T21:09:40.582 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93472.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93472.log.gz 2024-08-25T21:09:40.582 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73859.log 2024-08-25T21:09:40.582 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139408.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97835.log 2024-08-25T21:09:40.583 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139408.log.gz 2024-08-25T21:09:40.584 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73859.log.gz 2024-08-25T21:09:40.584 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72421.log 2024-08-25T21:09:40.584 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97835.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97835.log.gz 2024-08-25T21:09:40.584 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25273.log 2024-08-25T21:09:40.585 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72421.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72421.log.gz 2024-08-25T21:09:40.585 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122866.log 2024-08-25T21:09:40.586 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25273.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25273.log.gz 2024-08-25T21:09:40.586 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35660.log 2024-08-25T21:09:40.587 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122866.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122866.log.gz 2024-08-25T21:09:40.587 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121333.log 2024-08-25T21:09:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35660.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35660.log.gz 2024-08-25T21:09:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50750.log 2024-08-25T21:09:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121333.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121333.log.gz 2024-08-25T21:09:40.589 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70863.log 2024-08-25T21:09:40.589 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50750.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50750.log.gz 2024-08-25T21:09:40.589 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43895.log 2024-08-25T21:09:40.591 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70863.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70863.log.gz 2024-08-25T21:09:40.591 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82015.log 2024-08-25T21:09:40.591 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43895.log.gz 2024-08-25T21:09:40.591 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128387.log 2024-08-25T21:09:40.592 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82015.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82015.log.gz 2024-08-25T21:09:40.592 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52881.log 2024-08-25T21:09:40.592 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.128387.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128387.log.gz 2024-08-25T21:09:40.593 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51738.log 2024-08-25T21:09:40.593 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52881.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92973.log 2024-08-25T21:09:40.593 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52881.log.gz 2024-08-25T21:09:40.594 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51738.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51738.log.gz 2024-08-25T21:09:40.594 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27451.log 2024-08-25T21:09:40.595 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92973.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92973.log.gz 2024-08-25T21:09:40.595 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74818.log 2024-08-25T21:09:40.596 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27451.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134171.log 2024-08-25T21:09:40.596 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27451.log.gz 2024-08-25T21:09:40.597 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74818.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.74818.log.gz -- 2024-08-25T21:09:40.597 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.101933.log 2024-08-25T21:09:40.598 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134171.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134171.log.gz 2024-08-25T21:09:40.598 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134295.log 2024-08-25T21:09:40.599 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101933.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101933.log.gz 2024-08-25T21:09:40.599 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90076.log 2024-08-25T21:09:40.600 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134295.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32740.log 2024-08-25T21:09:40.600 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134295.log.gz 2024-08-25T21:09:40.600 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90076.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90076.log.gz 2024-08-25T21:09:40.600 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105258.log 2024-08-25T21:09:40.601 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32740.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32740.log.gzgzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86778.log 2024-08-25T21:09:40.601 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:40.602 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39542.log 2024-08-25T21:09:40.602 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105258.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105258.log.gz 2024-08-25T21:09:40.602 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86778.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86778.log.gz 2024-08-25T21:09:40.602 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28018.log 2024-08-25T21:09:40.603 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39542.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140352.log 2024-08-25T21:09:40.603 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39542.log.gz 2024-08-25T21:09:40.604 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28018.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28018.log.gz -5 2024-08-25T21:09:40.604 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.141376.log 2024-08-25T21:09:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.140352.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140352.log.gz 2024-08-25T21:09:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121462.log 2024-08-25T21:09:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141376.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141376.log.gz 2024-08-25T21:09:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121539.log 2024-08-25T21:09:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121462.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121462.log.gz 2024-08-25T21:09:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54259.log 2024-08-25T21:09:40.607 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121539.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39585.log 2024-08-25T21:09:40.607 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121539.log.gz 2024-08-25T21:09:40.608 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54259.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54259.log.gz 2024-08-25T21:09:40.608 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111934.log 2024-08-25T21:09:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39585.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39585.log.gz 2024-08-25T21:09:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.21646.log 2024-08-25T21:09:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111934.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111934.log.gz 2024-08-25T21:09:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105525.log 2024-08-25T21:09:40.610 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.21646.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster2-client.admin.21646.log.gz 2024-08-25T21:09:40.610 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.28622.log 2024-08-25T21:09:40.611 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105525.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105525.log.gz 2024-08-25T21:09:40.611 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119653.log 2024-08-25T21:09:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28622.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28622.log.gz 2024-08-25T21:09:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109623.log 2024-08-25T21:09:40.613 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119653.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119653.log.gz 2024-08-25T21:09:40.613 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121991.log 2024-08-25T21:09:40.614 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109623.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65835.log 2024-08-25T21:09:40.614 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109623.log.gz 2024-08-25T21:09:40.614 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121991.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.121991.log.gz -5 2024-08-25T21:09:40.615 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.122197.log 2024-08-25T21:09:40.615 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65835.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65835.log.gz 2024-08-25T21:09:40.615 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124866.log 2024-08-25T21:09:40.616 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122197.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40425.log 2024-08-25T21:09:40.616 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122197.log.gz 2024-08-25T21:09:40.617 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124866.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124866.log.gz 2024-08-25T21:09:40.617 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124295.log 2024-08-25T21:09:40.618 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40425.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40425.log.gz 2024-08-25T21:09:40.618 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136502.log 2024-08-25T21:09:40.618 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124295.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115049.log 2024-08-25T21:09:40.618 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124295.log.gz 2024-08-25T21:09:40.619 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136502.log.gz 2024-08-25T21:09:40.619 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27639.log 2024-08-25T21:09:40.620 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115049.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.115049.log.gz 2024-08-25T21:09:40.620 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76250.log 2024-08-25T21:09:40.621 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27639.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.61716.log 2024-08-25T21:09:40.621 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.27639.log.gz 2024-08-25T21:09:40.622 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76250.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76250.log.gz 2024-08-25T21:09:40.622 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31006.log 2024-08-25T21:09:40.622 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61716.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.61716.log.gz -5 2024-08-25T21:09:40.622 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.95601.log 2024-08-25T21:09:40.623 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31006.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31006.log.gz 2024-08-25T21:09:40.623 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53839.log 2024-08-25T21:09:40.624 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95601.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76720.log 2024-08-25T21:09:40.624 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95601.log.gz 2024-08-25T21:09:40.625 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53839.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53839.log.gz 2024-08-25T21:09:40.625 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95576.log 2024-08-25T21:09:40.626 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76720.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56543.log 2024-08-25T21:09:40.626 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76720.log.gz 2024-08-25T21:09:40.627 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95576.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95576.log.gz 2024-08-25T21:09:40.627 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128230.log 2024-08-25T21:09:40.628 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56543.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30852.log 2024-08-25T21:09:40.628 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56543.log.gz 2024-08-25T21:09:40.629 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128230.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128230.log.gz 2024-08-25T21:09:40.629 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88396.log 2024-08-25T21:09:40.629 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30852.log.gz 2024-08-25T21:09:40.629 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70760.log 2024-08-25T21:09:40.630 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88396.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88396.log.gz 2024-08-25T21:09:40.630 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127748.log 2024-08-25T21:09:40.631 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70760.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70760.log.gz 2024-08-25T21:09:40.631 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102437.log 2024-08-25T21:09:40.632 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127748.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127748.log.gz 2024-08-25T21:09:40.632 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122662.log 2024-08-25T21:09:40.633 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102437.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102437.log.gz 2024-08-25T21:09:40.633 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39919.log 2024-08-25T21:09:40.634 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122662.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92460.log 2024-08-25T21:09:40.634 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122662.log.gz 2024-08-25T21:09:40.634 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39919.log.gz 2024-08-25T21:09:40.634 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64322.log 2024-08-25T21:09:40.635 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.92460.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.92460.log.gz 2024-08-25T21:09:40.635 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66044.log 2024-08-25T21:09:40.636 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64322.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64322.log.gz 2024-08-25T21:09:40.636 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103781.log 2024-08-25T21:09:40.637 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66044.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66044.log.gz 2024-08-25T21:09:40.637 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27554.log 2024-08-25T21:09:40.638 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103781.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98718.log 2024-08-25T21:09:40.638 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103781.log.gz 2024-08-25T21:09:40.639 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27554.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27554.log.gz 2024-08-25T21:09:40.639 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77110.log 2024-08-25T21:09:40.639 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98718.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23291.log 2024-08-25T21:09:40.639 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98718.log.gz 2024-08-25T21:09:40.640 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77110.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77110.log.gz 2024-08-25T21:09:40.640 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135132.log 2024-08-25T21:09:40.641 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23291.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23291.log.gz 2024-08-25T21:09:40.641 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59404.log 2024-08-25T21:09:40.642 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135132.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135132.log.gz 2024-08-25T21:09:40.642 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104506.log 2024-08-25T21:09:40.642 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59404.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59404.log.gz 2024-08-25T21:09:40.643 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41769.log 2024-08-25T21:09:40.643 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.104506.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104506.log.gz 2024-08-25T21:09:40.644 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65666.log 2024-08-25T21:09:40.644 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41769.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.41769.log.gz 2024-08-25T21:09:40.644 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.81377.log 2024-08-25T21:09:40.645 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65666.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65666.log.gz 2024-08-25T21:09:40.645 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71220.log 2024-08-25T21:09:40.646 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81377.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88204.log 2024-08-25T21:09:40.646 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81377.log.gz 2024-08-25T21:09:40.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71220.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71220.log.gz 2024-08-25T21:09:40.647 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118996.log 2024-08-25T21:09:40.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88204.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88204.log.gz 2024-08-25T21:09:40.648 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71089.log 2024-08-25T21:09:40.648 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118996.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62449.log 2024-08-25T21:09:40.648 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118996.log.gz 2024-08-25T21:09:40.649 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129140.log 2024-08-25T21:09:40.649 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71089.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71089.log.gz 2024-08-25T21:09:40.650 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62449.log.gz 2024-08-25T21:09:40.650 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109024.log 2024-08-25T21:09:40.651 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.129140.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129140.log.gz 2024-08-25T21:09:40.651 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77721.log 2024-08-25T21:09:40.651 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109024.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109024.log.gz 2024-08-25T21:09:40.652 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85716.log 2024-08-25T21:09:40.652 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77721.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133803.log 2024-08-25T21:09:40.652 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77721.log.gz 2024-08-25T21:09:40.653 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85716.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85716.log.gz 2024-08-25T21:09:40.653 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31808.log 2024-08-25T21:09:40.654 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133803.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83291.log 2024-08-25T21:09:40.654 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133803.log.gz 2024-08-25T21:09:40.655 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31808.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31808.log.gz 2024-08-25T21:09:40.655 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72987.log 2024-08-25T21:09:40.655 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.83291.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83291.log.gz 2024-08-25T21:09:40.656 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70029.log 2024-08-25T21:09:40.656 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72987.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42145.log 2024-08-25T21:09:40.656 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72987.log.gz 2024-08-25T21:09:40.657 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70029.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70029.log.gz 2024-08-25T21:09:40.657 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91967.log 2024-08-25T21:09:40.658 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42145.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42145.log.gz 2024-08-25T21:09:40.658 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80470.log 2024-08-25T21:09:40.659 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91967.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91967.log.gz 2024-08-25T21:09:40.659 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41181.log 2024-08-25T21:09:40.660 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80470.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80470.log.gz 2024-08-25T21:09:40.660 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45521.log 2024-08-25T21:09:40.660 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41181.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41181.log.gz 2024-08-25T21:09:40.660 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102502.log 2024-08-25T21:09:40.661 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.45521.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45521.log.gz 2024-08-25T21:09:40.661 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122302.log 2024-08-25T21:09:40.662 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102502.log.gz 2024-08-25T21:09:40.662 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71042.log 2024-08-25T21:09:40.663 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122302.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122302.log.gz 2024-08-25T21:09:40.663 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57037.log 2024-08-25T21:09:40.664 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.71042.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71042.log.gz 2024-08-25T21:09:40.664 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64532.log 2024-08-25T21:09:40.665 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.57037.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57037.log.gz 2024-08-25T21:09:40.665 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80346.log 2024-08-25T21:09:40.665 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64532.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64532.log.gz 2024-08-25T21:09:40.666 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39837.log 2024-08-25T21:09:40.667 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80346.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80346.log.gz 2024-08-25T21:09:40.667 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89740.log 2024-08-25T21:09:40.668 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39837.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39837.log.gz 2024-08-25T21:09:40.668 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34485.log 2024-08-25T21:09:40.668 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89740.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89740.log.gz 2024-08-25T21:09:40.668 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58794.log 2024-08-25T21:09:40.669 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34485.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34485.log.gz 2024-08-25T21:09:40.669 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74906.log 2024-08-25T21:09:40.670 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58794.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58794.log.gz 2024-08-25T21:09:40.670 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-mon.a.log 2024-08-25T21:09:40.671 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74906.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74906.log.gz 2024-08-25T21:09:40.671 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58399.log 2024-08-25T21:09:40.671 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-mon.a.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32094.log 2024-08-25T21:09:40.672 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58399.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58399.log.gz 2024-08-25T21:09:40.672 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74421.log 2024-08-25T21:09:40.673 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32094.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32094.log.gz 2024-08-25T21:09:40.673 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117831.log 2024-08-25T21:09:40.674 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74421.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74421.log.gz 2024-08-25T21:09:40.674 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41265.log 2024-08-25T21:09:40.674 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.117831.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117831.log.gz 2024-08-25T21:09:40.675 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87597.log 2024-08-25T21:09:40.676 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41265.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41265.log.gz 2024-08-25T21:09:40.676 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121753.log 2024-08-25T21:09:40.676 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87597.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87597.log.gz 2024-08-25T21:09:40.676 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54826.log 2024-08-25T21:09:40.677 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121753.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114586.log 2024-08-25T21:09:40.677 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121753.log.gz 2024-08-25T21:09:40.678 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54826.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54826.log.gz 2024-08-25T21:09:40.678 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25504.log 2024-08-25T21:09:40.678 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114586.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114586.log.gz 2024-08-25T21:09:40.679 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39670.log 2024-08-25T21:09:40.679 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25504.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25504.log.gz 2024-08-25T21:09:40.679 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121103.log 2024-08-25T21:09:40.680 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39670.log.gz 2024-08-25T21:09:40.680 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106334.log 2024-08-25T21:09:40.681 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121103.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121103.log.gz 2024-08-25T21:09:40.681 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74422.log 2024-08-25T21:09:40.682 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.106334.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68041.log 2024-08-25T21:09:40.682 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106334.log.gz 2024-08-25T21:09:40.682 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74422.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74422.log.gz 2024-08-25T21:09:40.683 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31961.log 2024-08-25T21:09:40.683 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68041.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68041.log.gz 2024-08-25T21:09:40.683 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79900.log 2024-08-25T21:09:40.685 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31961.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31961.log.gz 2024-08-25T21:09:40.685 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65076.log 2024-08-25T21:09:40.685 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79900.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79900.log.gz 2024-08-25T21:09:40.685 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103318.log 2024-08-25T21:09:40.686 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65076.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65076.log.gz 2024-08-25T21:09:40.686 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123249.log 2024-08-25T21:09:40.686 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103318.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.103318.log.gz --verbose 2024-08-25T21:09:40.686 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.113695.log 2024-08-25T21:09:40.687 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123249.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123249.log.gz 2024-08-25T21:09:40.687 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37003.log 2024-08-25T21:09:40.688 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113695.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.113695.log.gz 2024-08-25T21:09:40.688 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.45813.log 2024-08-25T21:09:40.689 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37003.log.gz 2024-08-25T21:09:40.689 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108179.log 2024-08-25T21:09:40.689 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45813.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45813.log.gz 2024-08-25T21:09:40.689 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96060.log 2024-08-25T21:09:40.690 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108179.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108179.log.gz 2024-08-25T21:09:40.690 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85536.log 2024-08-25T21:09:40.691 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96060.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88566.log 2024-08-25T21:09:40.692 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96060.log.gz 2024-08-25T21:09:40.692 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85536.log.gz 2024-08-25T21:09:40.692 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.36225.log 2024-08-25T21:09:40.693 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88566.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.admin.102588.log -- replaced with /var/log/ceph/cluster1-client.mirror.88566.log.gz 2024-08-25T21:09:40.693 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:40.693 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.36225.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.36225.log.gz 2024-08-25T21:09:40.693 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50648.log 2024-08-25T21:09:40.694 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102588.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102588.log.gz 2024-08-25T21:09:40.694 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107679.log 2024-08-25T21:09:40.695 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50648.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50648.log.gz 2024-08-25T21:09:40.695 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60148.log 2024-08-25T21:09:40.696 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107679.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107679.log.gz 2024-08-25T21:09:40.696 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73833.log 2024-08-25T21:09:40.696 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60148.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60148.log.gz 2024-08-25T21:09:40.696 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66559.log 2024-08-25T21:09:40.697 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73833.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.54301.log 2024-08-25T21:09:40.697 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.73833.log.gz 2024-08-25T21:09:40.698 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66559.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66559.log.gz 2024-08-25T21:09:40.698 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105919.log 2024-08-25T21:09:40.699 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54301.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54301.log.gz 2024-08-25T21:09:40.699 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98548.log 2024-08-25T21:09:40.699 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.105919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105919.log.gz 2024-08-25T21:09:40.699 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87786.log 2024-08-25T21:09:40.700 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98548.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98548.log.gz 2024-08-25T21:09:40.700 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110981.log 2024-08-25T21:09:40.701 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87786.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87786.log.gz 2024-08-25T21:09:40.702 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47257.log 2024-08-25T21:09:40.702 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110981.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110981.log.gz 2024-08-25T21:09:40.702 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109864.log 2024-08-25T21:09:40.702 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47257.log.gz 2024-08-25T21:09:40.703 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102279.log 2024-08-25T21:09:40.703 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109864.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.mirror.109864.log.gz /var/log/ceph/cluster1-client.mirror.76138.log 2024-08-25T21:09:40.703 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:40.704 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102279.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102279.log.gz 2024-08-25T21:09:40.704 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86948.log 2024-08-25T21:09:40.705 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76138.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125217.log 2024-08-25T21:09:40.705 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76138.log.gz 2024-08-25T21:09:40.706 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86948.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86948.log.gz 2024-08-25T21:09:40.706 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95385.log 2024-08-25T21:09:40.706 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125217.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125217.log.gz 2024-08-25T21:09:40.707 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75265.log 2024-08-25T21:09:40.707 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95385.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95385.log.gz 2024-08-25T21:09:40.707 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59247.log 2024-08-25T21:09:40.708 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75265.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75265.log.gz -5 2024-08-25T21:09:40.708 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.99288.log 2024-08-25T21:09:40.709 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59247.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59247.log.gz 2024-08-25T21:09:40.710 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99225.log 2024-08-25T21:09:40.710 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99288.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99288.log.gz 2024-08-25T21:09:40.710 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88990.log 2024-08-25T21:09:40.710 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99225.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68771.log 2024-08-25T21:09:40.711 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99225.log.gz 2024-08-25T21:09:40.711 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88990.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.88990.log.gz 2024-08-25T21:09:40.712 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106013.log 2024-08-25T21:09:40.713 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68771.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68771.log.gz 2024-08-25T21:09:40.713 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100859.log 2024-08-25T21:09:40.713 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106013.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106013.log.gz 2024-08-25T21:09:40.713 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76902.log 2024-08-25T21:09:40.714 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100859.log.gz 2024-08-25T21:09:40.714 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64742.log 2024-08-25T21:09:40.714 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76902.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76902.log.gz 2024-08-25T21:09:40.714 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121121.log 2024-08-25T21:09:40.715 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64742.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64742.log.gz 2024-08-25T21:09:40.715 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124675.log 2024-08-25T21:09:40.716 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121121.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121121.log.gz 2024-08-25T21:09:40.716 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123814.log 2024-08-25T21:09:40.717 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124675.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.124675.log.gz 2024-08-25T21:09:40.717 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.116245.log 2024-08-25T21:09:40.718 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123814.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123814.log.gz 2024-08-25T21:09:40.718 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41391.log 2024-08-25T21:09:40.718 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116245.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116245.log.gz 2024-08-25T21:09:40.718 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84013.log 2024-08-25T21:09:40.719 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41391.log.gz 2024-08-25T21:09:40.719 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31565.log 2024-08-25T21:09:40.720 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84013.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84013.log.gz 2024-08-25T21:09:40.720 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137257.log 2024-08-25T21:09:40.721 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31565.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31565.log.gz 2024-08-25T21:09:40.721 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44205.log 2024-08-25T21:09:40.722 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137257.log.gz 2024-08-25T21:09:40.722 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129771.log 2024-08-25T21:09:40.723 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44205.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44205.log.gz 2024-08-25T21:09:40.723 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27787.log 2024-08-25T21:09:40.723 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129771.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67913.log 2024-08-25T21:09:40.723 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129771.log.gz 2024-08-25T21:09:40.724 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27787.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137405.log 2024-08-25T21:09:40.724 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27787.log.gz 2024-08-25T21:09:40.725 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67913.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26231.log 2024-08-25T21:09:40.725 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67913.log.gz 2024-08-25T21:09:40.726 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137405.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46500.log 2024-08-25T21:09:40.726 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137405.log.gz 2024-08-25T21:09:40.726 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26231.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24612.log 2024-08-25T21:09:40.726 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26231.log.gz 2024-08-25T21:09:40.727 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46500.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46500.log.gz 2024-08-25T21:09:40.727 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97752.log 2024-08-25T21:09:40.728 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24612.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25212.log 2024-08-25T21:09:40.728 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24612.log.gz 2024-08-25T21:09:40.729 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97752.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97752.log.gzgzip 2024-08-25T21:09:40.729 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105088.log 2024-08-25T21:09:40.729 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25212.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25212.log.gz 2024-08-25T21:09:40.730 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49638.log 2024-08-25T21:09:40.730 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105088.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105088.log.gz 2024-08-25T21:09:40.730 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55951.log 2024-08-25T21:09:40.731 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49638.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49638.log.gz 2024-08-25T21:09:40.731 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33924.log 2024-08-25T21:09:40.732 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55951.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55951.log.gz 2024-08-25T21:09:40.732 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72693.log 2024-08-25T21:09:40.733 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33924.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97375.log 2024-08-25T21:09:40.733 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33924.log.gz 2024-08-25T21:09:40.734 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72693.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72693.log.gz 2024-08-25T21:09:40.734 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75512.log 2024-08-25T21:09:40.734 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97375.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97375.log.gz 2024-08-25T21:09:40.734 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47738.log 2024-08-25T21:09:40.735 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75512.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75512.log.gz 2024-08-25T21:09:40.735 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37400.log 2024-08-25T21:09:40.736 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47738.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47738.log.gz 2024-08-25T21:09:40.736 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92951.log 2024-08-25T21:09:40.736 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37400.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37400.log.gz 2024-08-25T21:09:40.737 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114915.log 2024-08-25T21:09:40.738 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92951.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115765.log 2024-08-25T21:09:40.738 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92951.log.gz 2024-08-25T21:09:40.738 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114915.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114915.log.gz 2024-08-25T21:09:40.738 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135390.log 2024-08-25T21:09:40.739 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115765.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115765.log.gz 2024-08-25T21:09:40.739 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125386.log 2024-08-25T21:09:40.740 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135390.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135390.log.gz 2024-08-25T21:09:40.740 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63859.log 2024-08-25T21:09:40.741 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125386.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125386.log.gz 2024-08-25T21:09:40.741 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42889.log 2024-08-25T21:09:40.742 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63859.log.gz 2024-08-25T21:09:40.742 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136963.log 2024-08-25T21:09:40.743 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42889.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42889.log.gz 2024-08-25T21:09:40.743 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44620.log 2024-08-25T21:09:40.744 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136963.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136963.log.gz 2024-08-25T21:09:40.744 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48969.log 2024-08-25T21:09:40.744 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.44620.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44620.log.gz 2024-08-25T21:09:40.744 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138944.log 2024-08-25T21:09:40.745 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48969.log.gz 2024-08-25T21:09:40.745 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137700.log 2024-08-25T21:09:40.746 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138944.log.gz 2024-08-25T21:09:40.746 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54961.log 2024-08-25T21:09:40.747 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137700.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137700.log.gz 2024-08-25T21:09:40.747 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75326.log 2024-08-25T21:09:40.748 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54961.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54961.log.gz 2024-08-25T21:09:40.748 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141185.log 2024-08-25T21:09:40.748 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75326.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75326.log.gz 2024-08-25T21:09:40.748 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75892.log 2024-08-25T21:09:40.749 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141185.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141185.log.gz 2024-08-25T21:09:40.749 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119174.log 2024-08-25T21:09:40.749 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75892.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.75892.log.gz /var/log/ceph/cluster1-client.admin.119916.log 2024-08-25T21:09:40.749 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:40.750 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119174.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119174.log.gz 2024-08-25T21:09:40.750 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112309.log 2024-08-25T21:09:40.751 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.119916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.119916.log.gz 2024-08-25T21:09:40.751 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142201.log 2024-08-25T21:09:40.752 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112309.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112309.log.gz 2024-08-25T21:09:40.752 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92864.log 2024-08-25T21:09:40.753 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.142201.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142201.log.gz 2024-08-25T21:09:40.753 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88499.log 2024-08-25T21:09:40.754 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92864.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92864.log.gz 2024-08-25T21:09:40.754 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25168.log 2024-08-25T21:09:40.754 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88499.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88499.log.gz 2024-08-25T21:09:40.755 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63490.log 2024-08-25T21:09:40.755 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25168.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25168.log.gz 2024-08-25T21:09:40.755 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95712.log 2024-08-25T21:09:40.756 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63490.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63490.log.gz 2024-08-25T21:09:40.756 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114312.log 2024-08-25T21:09:40.757 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95712.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95712.log.gz 2024-08-25T21:09:40.757 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61128.log 2024-08-25T21:09:40.758 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114312.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114312.log.gz 2024-08-25T21:09:40.758 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72842.log 2024-08-25T21:09:40.759 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61128.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61128.log.gz 2024-08-25T21:09:40.759 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115475.log 2024-08-25T21:09:40.760 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72842.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72842.log.gz 2024-08-25T21:09:40.760 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49680.log 2024-08-25T21:09:40.760 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115475.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115475.log.gz 2024-08-25T21:09:40.761 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59842.log 2024-08-25T21:09:40.761 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49680.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49680.log.gz 2024-08-25T21:09:40.761 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63322.log 2024-08-25T21:09:40.762 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59842.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59842.log.gz 2024-08-25T21:09:40.762 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96829.log 2024-08-25T21:09:40.763 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63322.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63322.log.gz 2024-08-25T21:09:40.763 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36309.log 2024-08-25T21:09:40.764 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96829.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96829.log.gz 2024-08-25T21:09:40.764 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26125.log 2024-08-25T21:09:40.764 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36309.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36309.log.gz 2024-08-25T21:09:40.764 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52790.log 2024-08-25T21:09:40.765 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26125.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26125.log.gz 2024-08-25T21:09:40.765 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126860.log 2024-08-25T21:09:40.766 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52790.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.52790.log.gz 2024-08-25T21:09:40.766 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63245.log 2024-08-25T21:09:40.767 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126860.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126860.log.gz 2024-08-25T21:09:40.767 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100795.log 2024-08-25T21:09:40.768 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63245.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63245.log.gz 2024-08-25T21:09:40.768 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54216.log 2024-08-25T21:09:40.768 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100795.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100795.log.gz 2024-08-25T21:09:40.768 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40467.log 2024-08-25T21:09:40.769 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54216.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54216.log.gz 2024-08-25T21:09:40.769 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63354.log 2024-08-25T21:09:40.770 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40467.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40467.log.gz 2024-08-25T21:09:40.770 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63314.log 2024-08-25T21:09:40.771 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63354.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63354.log.gz 2024-08-25T21:09:40.771 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44933.log 2024-08-25T21:09:40.772 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63314.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63314.log.gz 2024-08-25T21:09:40.772 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.142473.log 2024-08-25T21:09:40.772 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44933.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44933.log.gz 2024-08-25T21:09:40.773 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119826.log 2024-08-25T21:09:40.773 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.142473.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.142473.log.gz 2024-08-25T21:09:40.773 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131809.log 2024-08-25T21:09:40.774 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119826.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119826.log.gz 2024-08-25T21:09:40.774 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88443.log 2024-08-25T21:09:40.775 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131809.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131809.log.gz 2024-08-25T21:09:40.775 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50562.log 2024-08-25T21:09:40.776 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88443.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88443.log.gz 2024-08-25T21:09:40.776 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99097.log 2024-08-25T21:09:40.776 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50562.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131789.log 2024-08-25T21:09:40.776 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50562.log.gz 2024-08-25T21:09:40.777 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99097.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99097.log.gz 2024-08-25T21:09:40.777 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42063.log 2024-08-25T21:09:40.778 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131789.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131789.log.gz 2024-08-25T21:09:40.778 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38913.log 2024-08-25T21:09:40.779 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42063.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120467.log 2024-08-25T21:09:40.779 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42063.log.gz 2024-08-25T21:09:40.779 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38913.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38913.log.gz 2024-08-25T21:09:40.780 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38492.log 2024-08-25T21:09:40.780 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120467.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120467.log.gz 2024-08-25T21:09:40.780 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.120108.log 2024-08-25T21:09:40.781 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38492.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.38492.log.gz -- 2024-08-25T21:09:40.781 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.96343.log 2024-08-25T21:09:40.782 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.120108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.120108.log.gz 2024-08-25T21:09:40.782 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31807.log 2024-08-25T21:09:40.783 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96343.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63858.log 2024-08-25T21:09:40.783 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96343.log.gz 2024-08-25T21:09:40.783 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31807.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31807.log.gz 2024-08-25T21:09:40.784 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101616.log 2024-08-25T21:09:40.784 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63858.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63858.log.gz 2024-08-25T21:09:40.785 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124710.log 2024-08-25T21:09:40.785 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101616.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101616.log.gz 2024-08-25T21:09:40.785 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71740.log 2024-08-25T21:09:40.786 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124710.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124710.log.gz 2024-08-25T21:09:40.786 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127165.log 2024-08-25T21:09:40.787 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71740.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58358.log 2024-08-25T21:09:40.787 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71740.log.gz 2024-08-25T21:09:40.788 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127165.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127165.log.gz 2024-08-25T21:09:40.788 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47674.log 2024-08-25T21:09:40.788 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58358.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94098.log 2024-08-25T21:09:40.788 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58358.log.gz 2024-08-25T21:09:40.789 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47674.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47674.log.gz 2024-08-25T21:09:40.789 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88524.log 2024-08-25T21:09:40.790 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94098.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94098.log.gz 2024-08-25T21:09:40.790 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35813.log 2024-08-25T21:09:40.791 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88524.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88524.log.gz 2024-08-25T21:09:40.791 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77892.log 2024-08-25T21:09:40.792 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35813.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35813.log.gz 2024-08-25T21:09:40.792 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74674.log 2024-08-25T21:09:40.792 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77892.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77892.log.gz 2024-08-25T21:09:40.793 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123411.log 2024-08-25T21:09:40.793 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74674.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74674.log.gz 2024-08-25T21:09:40.793 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72549.log 2024-08-25T21:09:40.794 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123411.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41599.log 2024-08-25T21:09:40.794 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123411.log.gz 2024-08-25T21:09:40.795 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72549.log.gz 2024-08-25T21:09:40.795 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24549.log 2024-08-25T21:09:40.796 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41599.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.59427.log 2024-08-25T21:09:40.796 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41599.log.gz 2024-08-25T21:09:40.797 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24549.log.gz 2024-08-25T21:09:40.797 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69695.log 2024-08-25T21:09:40.797 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.59427.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59427.log.gz 2024-08-25T21:09:40.798 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96975.log 2024-08-25T21:09:40.799 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69695.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69695.log.gz 2024-08-25T21:09:40.799 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71812.log 2024-08-25T21:09:40.800 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96975.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96975.log.gz 2024-08-25T21:09:40.800 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37317.log 2024-08-25T21:09:40.800 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71812.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71812.log.gz 2024-08-25T21:09:40.800 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126517.log 2024-08-25T21:09:40.801 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37317.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37317.log.gz 2024-08-25T21:09:40.801 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118214.log 2024-08-25T21:09:40.802 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126517.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126517.log.gz 2024-08-25T21:09:40.802 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.93911.log 2024-08-25T21:09:40.803 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118214.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118214.log.gz 2024-08-25T21:09:40.803 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26421.log 2024-08-25T21:09:40.804 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.93911.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85138.log 2024-08-25T21:09:40.804 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.93911.log.gz 2024-08-25T21:09:40.804 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26421.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26421.log.gz 2024-08-25T21:09:40.805 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93109.log 2024-08-25T21:09:40.805 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85138.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85138.log.gz 2024-08-25T21:09:40.805 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50394.log 2024-08-25T21:09:40.806 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.93109.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93109.log.gz 2024-08-25T21:09:40.806 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134032.log 2024-08-25T21:09:40.807 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50394.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125782.log 2024-08-25T21:09:40.807 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50394.log.gz 2024-08-25T21:09:40.808 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134032.log.gz 2024-08-25T21:09:40.808 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46655.log 2024-08-25T21:09:40.809 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125782.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125782.log.gz 2024-08-25T21:09:40.809 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121840.log 2024-08-25T21:09:40.809 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.46655.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46655.log.gz 2024-08-25T21:09:40.810 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92745.log 2024-08-25T21:09:40.810 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121840.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121840.log.gz 2024-08-25T21:09:40.811 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136124.log 2024-08-25T21:09:40.811 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92745.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92745.log.gz 2024-08-25T21:09:40.811 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27429.log 2024-08-25T21:09:40.812 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136124.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113704.log 2024-08-25T21:09:40.812 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136124.log.gz 2024-08-25T21:09:40.813 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27429.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27429.log.gz 2024-08-25T21:09:40.813 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39963.log 2024-08-25T21:09:40.814 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113704.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113704.log.gz 2024-08-25T21:09:40.814 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54872.log 2024-08-25T21:09:40.815 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39963.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94326.log 2024-08-25T21:09:40.815 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39963.log.gz 2024-08-25T21:09:40.816 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54872.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54872.log.gz 2024-08-25T21:09:40.816 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76961.log 2024-08-25T21:09:40.817 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94326.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94326.log.gz 2024-08-25T21:09:40.817 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134971.log 2024-08-25T21:09:40.817 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76961.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76961.log.gz 2024-08-25T21:09:40.817 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134802.log 2024-08-25T21:09:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134971.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134971.log.gz 2024-08-25T21:09:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113777.log 2024-08-25T21:09:40.819 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134802.log.gz 2024-08-25T21:09:40.819 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.142759.log 2024-08-25T21:09:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113777.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113777.log.gz 2024-08-25T21:09:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46956.log 2024-08-25T21:09:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.142759.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.142759.log.gz 2024-08-25T21:09:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127828.log 2024-08-25T21:09:40.821 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46956.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46956.log.gz 2024-08-25T21:09:40.821 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27221.log 2024-08-25T21:09:40.822 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.127828.log: gzip 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127828.log.gz -5 2024-08-25T21:09:40.822 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.48343.log 2024-08-25T21:09:40.822 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27221.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27221.log.gz 2024-08-25T21:09:40.823 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136425.log 2024-08-25T21:09:40.823 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48343.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48343.log.gz 2024-08-25T21:09:40.823 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63608.log 2024-08-25T21:09:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136425.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136425.log.gz 2024-08-25T21:09:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62405.log 2024-08-25T21:09:40.825 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63608.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63608.log.gz 2024-08-25T21:09:40.825 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105946.log 2024-08-25T21:09:40.826 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62405.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62405.log.gz 2024-08-25T21:09:40.826 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58244.log 2024-08-25T21:09:40.827 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.105946.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105946.log.gz 2024-08-25T21:09:40.827 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80449.log 2024-08-25T21:09:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58244.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58244.log.gz 2024-08-25T21:09:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97709.log 2024-08-25T21:09:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.80449.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80449.log.gz 2024-08-25T21:09:40.829 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128401.log 2024-08-25T21:09:40.829 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97709.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97709.log.gz 2024-08-25T21:09:40.829 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116971.log 2024-08-25T21:09:40.830 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128401.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128401.log.gz 2024-08-25T21:09:40.830 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135852.log 2024-08-25T21:09:40.831 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.116971.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123772.log 2024-08-25T21:09:40.831 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.116971.log.gz 2024-08-25T21:09:40.832 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135852.log.gz 2024-08-25T21:09:40.832 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54016.log 2024-08-25T21:09:40.832 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123772.log.gz 2024-08-25T21:09:40.833 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86695.log 2024-08-25T21:09:40.833 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54016.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54016.log.gz 2024-08-25T21:09:40.833 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108093.log 2024-08-25T21:09:40.834 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86695.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86695.log.gz 2024-08-25T21:09:40.834 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38177.log 2024-08-25T21:09:40.835 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108093.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129657.log 2024-08-25T21:09:40.836 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108093.log.gz 2024-08-25T21:09:40.836 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38177.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38177.log.gz 2024-08-25T21:09:40.836 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125074.log 2024-08-25T21:09:40.836 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129657.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129657.log.gz 2024-08-25T21:09:40.836 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75122.log 2024-08-25T21:09:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125074.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125074.log.gz 2024-08-25T21:09:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44306.log 2024-08-25T21:09:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75122.log.gz 2024-08-25T21:09:40.838 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43189.log 2024-08-25T21:09:40.839 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44306.log.gz 2024-08-25T21:09:40.839 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42397.log 2024-08-25T21:09:40.839 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43189.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43189.log.gz 2024-08-25T21:09:40.839 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23688.log 2024-08-25T21:09:40.840 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42397.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42397.log.gz 2024-08-25T21:09:40.840 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42912.log 2024-08-25T21:09:40.841 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23688.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.23688.log.gz -5 2024-08-25T21:09:40.841 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.22822.log 2024-08-25T21:09:40.842 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.42912.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42912.log.gz 2024-08-25T21:09:40.842 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80230.log 2024-08-25T21:09:40.843 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.22822.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.22822.log.gz 2024-08-25T21:09:40.843 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74996.log 2024-08-25T21:09:40.844 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.80230.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80230.log.gz 2024-08-25T21:09:40.844 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96659.log 2024-08-25T21:09:40.844 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74996.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74996.log.gz -5 2024-08-25T21:09:40.844 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.53754.log 2024-08-25T21:09:40.845 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96659.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96659.log.gz 2024-08-25T21:09:40.845 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57923.log 2024-08-25T21:09:40.846 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53754.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53754.log.gz 2024-08-25T21:09:40.846 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66545.log 2024-08-25T21:09:40.847 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.57923.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.57923.log.gz 2024-08-25T21:09:40.847 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136714.log 2024-08-25T21:09:40.848 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66545.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66545.log.gz 2024-08-25T21:09:40.848 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57199.log 2024-08-25T21:09:40.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.2 has been restored 2024-08-25T21:09:40.849 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136714.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136714.log.gz 2024-08-25T21:09:40.849 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84995.log 2024-08-25T21:09:40.849 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57199.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57199.log.gz 2024-08-25T21:09:40.850 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101696.log 2024-08-25T21:09:40.850 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84995.log.gz 2024-08-25T21:09:40.850 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23115.log 2024-08-25T21:09:40.851 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101696.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101696.log.gz 2024-08-25T21:09:40.851 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30935.log 2024-08-25T21:09:40.852 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23115.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29233.log 2024-08-25T21:09:40.852 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23115.log.gz 2024-08-25T21:09:40.853 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.30935.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30935.log.gz 2024-08-25T21:09:40.853 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45084.log 2024-08-25T21:09:40.854 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29233.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29233.log.gz 2024-08-25T21:09:40.854 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137721.log 2024-08-25T21:09:40.854 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.45084.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45084.log.gz 2024-08-25T21:09:40.855 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78648.log 2024-08-25T21:09:40.855 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137721.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137721.log.gz 2024-08-25T21:09:40.855 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83243.log 2024-08-25T21:09:40.856 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78648.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78648.log.gz 2024-08-25T21:09:40.856 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124336.log 2024-08-25T21:09:40.857 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83243.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83243.log.gz 2024-08-25T21:09:40.857 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141250.log 2024-08-25T21:09:40.857 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124336.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124336.log.gz 2024-08-25T21:09:40.858 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97327.log 2024-08-25T21:09:40.858 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141250.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141250.log.gz 2024-08-25T21:09:40.859 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108206.log 2024-08-25T21:09:40.859 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97327.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97327.log.gz 2024-08-25T21:09:40.859 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137114.log 2024-08-25T21:09:40.860 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108206.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108206.log.gz 2024-08-25T21:09:40.860 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116916.log 2024-08-25T21:09:40.861 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137114.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137114.log.gz 2024-08-25T21:09:40.861 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74969.log 2024-08-25T21:09:40.861 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.116916.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.116916.log.gz -5 2024-08-25T21:09:40.862 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.41851.log 2024-08-25T21:09:40.862 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74969.log.gz 2024-08-25T21:09:40.863 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98173.log 2024-08-25T21:09:40.864 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41851.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41851.log.gz 2024-08-25T21:09:40.864 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51194.log 2024-08-25T21:09:40.864 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98173.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98173.log.gz 2024-08-25T21:09:40.864 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63944.log 2024-08-25T21:09:40.865 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51194.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51194.log.gz 2024-08-25T21:09:40.865 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97378.log 2024-08-25T21:09:40.866 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63944.log.gz 2024-08-25T21:09:40.866 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107411.log 2024-08-25T21:09:40.867 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97378.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97378.log.gz 2024-08-25T21:09:40.867 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74212.log 2024-08-25T21:09:40.868 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107411.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107411.log.gz 2024-08-25T21:09:40.868 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.95998.log 2024-08-25T21:09:40.868 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74212.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74212.log.gz 2024-08-25T21:09:40.869 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102780.log 2024-08-25T21:09:40.869 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.95998.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95998.log.gz 2024-08-25T21:09:40.869 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69682.log 2024-08-25T21:09:40.870 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102780.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102780.log.gz 2024-08-25T21:09:40.870 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56064.log 2024-08-25T21:09:40.871 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69682.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69682.log.gz 2024-08-25T21:09:40.871 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137171.log 2024-08-25T21:09:40.872 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56064.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56064.log.gz 2024-08-25T21:09:40.872 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39753.log 2024-08-25T21:09:40.873 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137171.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137171.log.gz 2024-08-25T21:09:40.873 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50058.log 2024-08-25T21:09:40.873 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39753.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.39753.log.gz 2024-08-25T21:09:40.873 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89890.log 2024-08-25T21:09:40.874 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50058.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50058.log.gz 2024-08-25T21:09:40.874 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91471.log 2024-08-25T21:09:40.875 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89890.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89890.log.gz 2024-08-25T21:09:40.875 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59273.log 2024-08-25T21:09:40.876 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91471.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91471.log.gz -5 2024-08-25T21:09:40.876 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.67627.log 2024-08-25T21:09:40.876 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59273.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59273.log.gz 2024-08-25T21:09:40.876 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125783.log 2024-08-25T21:09:40.877 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67627.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67627.log.gz 2024-08-25T21:09:40.877 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62368.log 2024-08-25T21:09:40.878 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125783.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125783.log.gz 2024-08-25T21:09:40.878 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86021.log 2024-08-25T21:09:40.879 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62368.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62368.log.gz 2024-08-25T21:09:40.879 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.23862.log 2024-08-25T21:09:40.880 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86021.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86021.log.gz 2024-08-25T21:09:40.880 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30662.log 2024-08-25T21:09:40.880 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.23862.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.23862.log.gz 2024-08-25T21:09:40.881 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46447.log 2024-08-25T21:09:40.881 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30662.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30662.log.gz 2024-08-25T21:09:40.881 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56437.log 2024-08-25T21:09:40.882 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46447.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46447.log.gz 2024-08-25T21:09:40.882 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86696.log 2024-08-25T21:09:40.883 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56437.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56437.log.gz 2024-08-25T21:09:40.883 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60670.log 2024-08-25T21:09:40.884 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86696.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86696.log.gz 2024-08-25T21:09:40.884 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128311.log 2024-08-25T21:09:40.885 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60670.log.gz 2024-08-25T21:09:40.885 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34827.log 2024-08-25T21:09:40.886 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128311.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128311.log.gz 2024-08-25T21:09:40.886 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56650.log 2024-08-25T21:09:40.887 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34827.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34827.log.gz 2024-08-25T21:09:40.887 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37213.log 2024-08-25T21:09:40.888 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56650.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56650.log.gz 2024-08-25T21:09:40.888 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92752.log 2024-08-25T21:09:40.888 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37213.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37213.log.gz 2024-08-25T21:09:40.889 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110222.log 2024-08-25T21:09:40.889 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.92752.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.92752.log.gz 2024-08-25T21:09:40.890 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67119.log 2024-08-25T21:09:40.890 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110222.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110222.log.gz 2024-08-25T21:09:40.890 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38333.log 2024-08-25T21:09:40.891 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67119.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67119.log.gz 2024-08-25T21:09:40.891 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.33021.log 2024-08-25T21:09:40.892 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38333.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38333.log.gz 2024-08-25T21:09:40.892 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48864.log 2024-08-25T21:09:40.893 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.33021.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.33021.log.gz 2024-08-25T21:09:40.893 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44772.log 2024-08-25T21:09:40.894 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48864.log.gz 2024-08-25T21:09:40.895 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98717.log 2024-08-25T21:09:40.895 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44772.log.gz 2024-08-25T21:09:40.895 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98213.log 2024-08-25T21:09:40.896 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98717.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98717.log.gz 2024-08-25T21:09:40.896 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72201.log 2024-08-25T21:09:40.896 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98213.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98213.log.gz 2024-08-25T21:09:40.896 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91968.log 2024-08-25T21:09:40.897 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72201.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72201.log.gz 2024-08-25T21:09:40.897 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119739.log 2024-08-25T21:09:40.898 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91968.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.47295.log -- replaced with /var/log/ceph/cluster1-client.admin.91968.log.gz 2024-08-25T21:09:40.898 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:40.899 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119739.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119739.log.gz 2024-08-25T21:09:40.900 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103782.log 2024-08-25T21:09:40.900 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47295.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47295.log.gz 2024-08-25T21:09:40.900 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129698.log 2024-08-25T21:09:40.900 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103782.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103782.log.gz 2024-08-25T21:09:40.900 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108354.log 2024-08-25T21:09:40.901 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129698.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129698.log.gz 2024-08-25T21:09:40.901 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40927.log 2024-08-25T21:09:40.902 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108354.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108354.log.gz 2024-08-25T21:09:40.902 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65035.log 2024-08-25T21:09:40.903 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40927.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40927.log.gz 2024-08-25T21:09:40.903 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71965.log 2024-08-25T21:09:40.904 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65035.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65035.log.gz 2024-08-25T21:09:40.904 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127477.log 2024-08-25T21:09:40.904 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71965.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71965.log.gz 2024-08-25T21:09:40.904 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69915.log 2024-08-25T21:09:40.906 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127477.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127477.log.gz 2024-08-25T21:09:40.906 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81109.log 2024-08-25T21:09:40.906 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.69915.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.69915.log.gz 2024-08-25T21:09:40.906 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43938.log 2024-08-25T21:09:40.907 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81109.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81109.log.gz 2024-08-25T21:09:40.907 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107153.log 2024-08-25T21:09:40.907 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43938.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43938.log.gz 2024-08-25T21:09:40.907 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98800.log 2024-08-25T21:09:40.908 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107153.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107153.log.gz 2024-08-25T21:09:40.908 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37188.log 2024-08-25T21:09:40.909 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98800.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98800.log.gz 2024-08-25T21:09:40.909 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82813.log 2024-08-25T21:09:40.910 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37188.log.gz 2024-08-25T21:09:40.910 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80472.log 2024-08-25T21:09:40.911 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82813.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82813.log.gz 2024-08-25T21:09:40.911 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83205.log 2024-08-25T21:09:40.911 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80472.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80472.log.gz 2024-08-25T21:09:40.911 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141677.log 2024-08-25T21:09:40.912 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83205.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83205.log.gz 2024-08-25T21:09:40.913 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75830.log 2024-08-25T21:09:40.913 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141677.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141677.log.gz 2024-08-25T21:09:40.913 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127876.log 2024-08-25T21:09:40.913 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75830.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75830.log.gz 2024-08-25T21:09:40.914 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29782.log 2024-08-25T21:09:40.914 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.127876.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster2-client.admin.127876.log.gz --verbose 2024-08-25T21:09:40.914 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster2-client.mirror.130093.log 2024-08-25T21:09:40.915 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29782.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29782.log.gz 2024-08-25T21:09:40.915 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112333.log 2024-08-25T21:09:40.915 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.130093.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.130093.log.gz 2024-08-25T21:09:40.916 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28351.log 2024-08-25T21:09:40.916 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112333.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112333.log.gz 2024-08-25T21:09:40.917 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93498.log 2024-08-25T21:09:40.917 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28351.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28351.log.gz 2024-08-25T21:09:40.917 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130841.log 2024-08-25T21:09:40.918 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93498.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93498.log.gz 2024-08-25T21:09:40.918 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63566.log 2024-08-25T21:09:40.919 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.130841.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130841.log.gz -5 2024-08-25T21:09:40.919 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.133883.log 2024-08-25T21:09:40.920 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63566.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63566.log.gz 2024-08-25T21:09:40.920 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119691.log 2024-08-25T21:09:40.920 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133883.log.gz 2024-08-25T21:09:40.920 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109444.log 2024-08-25T21:09:40.921 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119691.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119691.log.gz 2024-08-25T21:09:40.921 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130958.log 2024-08-25T21:09:40.922 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109444.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109444.log.gz 2024-08-25T21:09:40.922 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73069.log 2024-08-25T21:09:40.922 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130958.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130958.log.gz 2024-08-25T21:09:40.923 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129999.log 2024-08-25T21:09:40.924 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73069.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73069.log.gz 2024-08-25T21:09:40.924 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48718.log 2024-08-25T21:09:40.924 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129999.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129999.log.gz 2024-08-25T21:09:40.924 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113214.log 2024-08-25T21:09:40.925 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48718.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48718.log.gz 2024-08-25T21:09:40.925 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98465.log 2024-08-25T21:09:40.927 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113214.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113214.log.gz 2024-08-25T21:09:40.927 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141170.log 2024-08-25T21:09:40.927 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98465.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98465.log.gz 2024-08-25T21:09:40.927 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.17304.log 2024-08-25T21:09:40.932 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141170.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141170.log.gz 2024-08-25T21:09:40.932 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51244.log 2024-08-25T21:09:40.932 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.17304.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.17304.log.gz 2024-08-25T21:09:40.933 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.138037.log 2024-08-25T21:09:40.934 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51244.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51244.log.gz 2024-08-25T21:09:40.934 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23229.log 2024-08-25T21:09:40.934 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.138037.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.138037.log.gz 2024-08-25T21:09:40.934 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49386.log 2024-08-25T21:09:40.934 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23229.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23229.log.gz 2024-08-25T21:09:40.934 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103561.log 2024-08-25T21:09:40.934 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49386.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91469.log 2024-08-25T21:09:40.934 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49386.log.gz 2024-08-25T21:09:40.937 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103561.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46474.log 2024-08-25T21:09:40.938 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103561.log.gz 2024-08-25T21:09:40.938 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91469.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91469.log.gz 2024-08-25T21:09:40.938 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86738.log 2024-08-25T21:09:40.938 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46474.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46474.log.gz 2024-08-25T21:09:40.938 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43806.log 2024-08-25T21:09:40.938 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86738.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86738.log.gz 2024-08-25T21:09:40.938 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89681.log 2024-08-25T21:09:40.938 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43806.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43806.log.gz 2024-08-25T21:09:40.938 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108091.log 2024-08-25T21:09:40.938 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89681.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89681.log.gz 2024-08-25T21:09:40.938 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45346.log 2024-08-25T21:09:40.938 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108091.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108091.log.gz 2024-08-25T21:09:40.939 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75681.log 2024-08-25T21:09:40.939 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.45346.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45346.log.gz 2024-08-25T21:09:40.939 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29361.log 2024-08-25T21:09:40.940 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75681.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75681.log.gz 2024-08-25T21:09:40.940 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30621.log 2024-08-25T21:09:40.940 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29361.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29361.log.gz 2024-08-25T21:09:40.941 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112834.log 2024-08-25T21:09:40.941 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30621.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30621.log.gz 2024-08-25T21:09:40.941 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130661.log 2024-08-25T21:09:40.941 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112834.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112834.log.gz 2024-08-25T21:09:40.941 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39501.log 2024-08-25T21:09:40.942 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130661.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130661.log.gz 2024-08-25T21:09:40.942 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107526.log 2024-08-25T21:09:40.943 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39501.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39501.log.gz 2024-08-25T21:09:40.943 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30074.log 2024-08-25T21:09:40.943 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107526.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107526.log.gz 2024-08-25T21:09:40.943 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37589.log 2024-08-25T21:09:40.944 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30074.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30074.log.gz 2024-08-25T21:09:40.944 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133131.log 2024-08-25T21:09:40.944 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37589.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37589.log.gz 2024-08-25T21:09:40.944 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136693.log 2024-08-25T21:09:40.945 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133131.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133131.log.gz 2024-08-25T21:09:40.945 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75175.log 2024-08-25T21:09:40.945 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136693.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136693.log.gz 2024-08-25T21:09:40.946 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96059.log 2024-08-25T21:09:40.946 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75175.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75175.log.gz 2024-08-25T21:09:40.946 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132354.log 2024-08-25T21:09:40.947 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96059.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89065.log 2024-08-25T21:09:40.947 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96059.log.gz 2024-08-25T21:09:40.948 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132354.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48719.log 2024-08-25T21:09:40.948 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132354.log.gz 2024-08-25T21:09:40.949 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89065.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89065.log.gz 2024-08-25T21:09:40.949 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52158.log 2024-08-25T21:09:40.949 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48719.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48719.log.gz 2024-08-25T21:09:40.950 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87850.log 2024-08-25T21:09:40.950 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52158.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52158.log.gz 2024-08-25T21:09:40.950 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69848.log 2024-08-25T21:09:40.951 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87850.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87850.log.gz 2024-08-25T21:09:40.951 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61357.log 2024-08-25T21:09:40.951 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.69848.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69848.log.gz 2024-08-25T21:09:40.952 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47521.log 2024-08-25T21:09:40.952 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61357.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61357.log.gz 2024-08-25T21:09:40.953 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43437.log 2024-08-25T21:09:40.954 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47521.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47521.log.gz 2024-08-25T21:09:40.954 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59115.log 2024-08-25T21:09:40.954 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43437.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43437.log.gz 2024-08-25T21:09:40.955 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62766.log 2024-08-25T21:09:40.955 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59115.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59115.log.gz 2024-08-25T21:09:40.956 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74528.log 2024-08-25T21:09:40.956 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62766.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62766.log.gz 2024-08-25T21:09:40.957 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54135.log 2024-08-25T21:09:40.957 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74528.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74528.log.gz 2024-08-25T21:09:40.957 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111995.log 2024-08-25T21:09:40.958 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54135.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81002.log 2024-08-25T21:09:40.958 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54135.log.gz 2024-08-25T21:09:40.959 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111995.log.gz 2024-08-25T21:09:40.959 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40718.log 2024-08-25T21:09:40.960 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81002.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81002.log.gz 2024-08-25T21:09:40.960 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123371.log 2024-08-25T21:09:40.960 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40718.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40718.log.gz 2024-08-25T21:09:40.961 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83573.log 2024-08-25T21:09:40.961 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123371.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123371.log.gz 2024-08-25T21:09:40.962 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135790.log 2024-08-25T21:09:40.962 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83573.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83573.log.gz 2024-08-25T21:09:40.963 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92482.log 2024-08-25T21:09:40.963 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135790.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135790.log.gz 2024-08-25T21:09:40.963 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29611.log 2024-08-25T21:09:40.964 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92482.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92482.log.gz 2024-08-25T21:09:40.964 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86737.log 2024-08-25T21:09:40.965 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29611.log.gz 2024-08-25T21:09:40.965 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129242.log 2024-08-25T21:09:40.966 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86737.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86737.log.gz 2024-08-25T21:09:40.966 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132146.log 2024-08-25T21:09:40.967 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129242.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129242.log.gz 2024-08-25T21:09:40.967 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31763.log 2024-08-25T21:09:40.968 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132146.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132146.log.gz 2024-08-25T21:09:40.968 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68204.log 2024-08-25T21:09:40.969 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31763.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31763.log.gz 2024-08-25T21:09:40.969 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112251.log 2024-08-25T21:09:40.970 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68204.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86675.log 2024-08-25T21:09:40.970 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68204.log.gz 2024-08-25T21:09:40.970 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112251.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84828.log 2024-08-25T21:09:40.970 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112251.log.gz 2024-08-25T21:09:40.971 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86675.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86675.log.gz 2024-08-25T21:09:40.971 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114034.log 2024-08-25T21:09:40.972 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84828.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135011.log 2024-08-25T21:09:40.972 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84828.log.gz 2024-08-25T21:09:40.973 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114034.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114034.log.gz 2024-08-25T21:09:40.973 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56304.log 2024-08-25T21:09:40.973 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135011.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40382.log 0.0% 2024-08-25T21:09:40.974 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.135011.log.gz 2024-08-25T21:09:40.974 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56304.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56304.log.gz 2024-08-25T21:09:40.975 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92077.log 2024-08-25T21:09:40.975 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40382.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40382.log.gz 2024-08-25T21:09:40.975 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61192.log 2024-08-25T21:09:40.976 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92077.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92077.log.gz 2024-08-25T21:09:40.976 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23359.log 2024-08-25T21:09:40.977 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61192.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125137.log 2024-08-25T21:09:40.977 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61192.log.gz 2024-08-25T21:09:40.978 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23359.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23359.log.gz 2024-08-25T21:09:40.978 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.22976.log 2024-08-25T21:09:40.979 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125137.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125137.log.gz 2024-08-25T21:09:40.979 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67319.log 2024-08-25T21:09:40.979 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.22976.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.22976.log.gz 2024-08-25T21:09:40.980 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121207.log 2024-08-25T21:09:40.980 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67319.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67319.log.gz 2024-08-25T21:09:40.980 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96910.log 2024-08-25T21:09:40.981 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121207.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121207.log.gz 2024-08-25T21:09:40.981 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.22426.log 2024-08-25T21:09:40.982 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96910.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96910.log.gz 2024-08-25T21:09:40.982 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45502.log 2024-08-25T21:09:40.983 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.0.22426.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25758.log 2024-08-25T21:09:40.984 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45502.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45037.log 2024-08-25T21:09:40.984 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45502.log.gz 2024-08-25T21:09:40.984 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25758.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25758.log.gz -5 2024-08-25T21:09:40.985 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.139938.log 2024-08-25T21:09:40.985 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.45037.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45037.log.gz 2024-08-25T21:09:40.985 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84322.log 2024-08-25T21:09:40.986 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.139938.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.139938.log.gz 2024-08-25T21:09:40.987 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27302.log 2024-08-25T21:09:40.987 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84322.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84322.log.gz 2024-08-25T21:09:40.988 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.31306.log 2024-08-25T21:09:40.988 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27302.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27302.log.gz 2024-08-25T21:09:40.988 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77078.log 2024-08-25T21:09:40.989 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.31306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.31306.log.gz 2024-08-25T21:09:40.989 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89054.log 2024-08-25T21:09:40.990 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77078.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77078.log.gz 2024-08-25T21:09:40.990 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97544.log 2024-08-25T21:09:40.990 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89054.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89054.log.gz 2024-08-25T21:09:40.991 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64706.log 2024-08-25T21:09:40.991 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97544.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97544.log.gz 2024-08-25T21:09:40.991 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132900.log 2024-08-25T21:09:40.992 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64706.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64706.log.gz 2024-08-25T21:09:40.992 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76313.log 2024-08-25T21:09:40.993 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132900.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132900.log.gz 2024-08-25T21:09:40.993 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90100.log 2024-08-25T21:09:40.993 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76313.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76313.log.gz 2024-08-25T21:09:40.994 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47343.log 2024-08-25T21:09:40.994 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90100.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90100.log.gz 2024-08-25T21:09:40.994 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131990.log 2024-08-25T21:09:40.995 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.47343.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47343.log.gz 2024-08-25T21:09:40.995 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78901.log 2024-08-25T21:09:40.996 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131990.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131990.log.gz 2024-08-25T21:09:40.996 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109506.log 2024-08-25T21:09:40.996 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78901.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78901.log.gz 2024-08-25T21:09:40.997 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39124.log 2024-08-25T21:09:40.997 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109506.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109506.log.gz 2024-08-25T21:09:40.997 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127230.log 2024-08-25T21:09:40.998 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39124.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39124.log.gz 2024-08-25T21:09:40.998 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104279.log 2024-08-25T21:09:40.999 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127230.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127230.log.gz 2024-08-25T21:09:40.999 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107221.log 2024-08-25T21:09:40.999 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104279.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104279.log.gz 2024-08-25T21:09:40.999 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90460.log 2024-08-25T21:09:41.000 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107221.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107221.log.gz 2024-08-25T21:09:41.000 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90033.log 2024-08-25T21:09:41.001 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90460.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90460.log.gz 2024-08-25T21:09:41.001 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32467.log 2024-08-25T21:09:41.001 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90033.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90033.log.gz 2024-08-25T21:09:41.002 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56335.log 2024-08-25T21:09:41.002 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32467.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93525.log 2024-08-25T21:09:41.002 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32467.log.gz 2024-08-25T21:09:41.003 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56335.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56335.log.gz 2024-08-25T21:09:41.003 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137404.log 2024-08-25T21:09:41.004 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93525.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93525.log.gz 2024-08-25T21:09:41.004 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110223.log 2024-08-25T21:09:41.005 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137404.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137404.log.gz 2024-08-25T21:09:41.005 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.128174.log 2024-08-25T21:09:41.005 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110223.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110223.log.gz 2024-08-25T21:09:41.006 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136247.log 2024-08-25T21:09:41.006 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.128174.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53084.log 2024-08-25T21:09:41.006 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.128174.log.gz 2024-08-25T21:09:41.007 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136247.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136247.log.gz 2024-08-25T21:09:41.007 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132564.log 2024-08-25T21:09:41.008 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53084.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53084.log.gz 2024-08-25T21:09:41.008 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116047.log 2024-08-25T21:09:41.008 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132564.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132564.log.gz 2024-08-25T21:09:41.008 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133068.log 2024-08-25T21:09:41.009 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.116047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.116047.log.gz 2024-08-25T21:09:41.009 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110696.log 2024-08-25T21:09:41.010 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133068.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133068.log.gz 2024-08-25T21:09:41.010 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36732.log 2024-08-25T21:09:41.011 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110696.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110696.log.gz 2024-08-25T21:09:41.011 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112692.log 2024-08-25T21:09:41.012 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36732.log.gz 2024-08-25T21:09:41.012 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113085.log 2024-08-25T21:09:41.013 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112692.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112692.log.gz 2024-08-25T21:09:41.013 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55314.log 2024-08-25T21:09:41.013 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113085.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.113085.log.gz --verbose 2024-08-25T21:09:41.014 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster2-client.admin.68675.log 2024-08-25T21:09:41.014 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55314.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55314.log.gz 2024-08-25T21:09:41.014 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24897.log 2024-08-25T21:09:41.015 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68675.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68675.log.gz 2024-08-25T21:09:41.015 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123033.log 2024-08-25T21:09:41.016 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24897.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24897.log.gz 2024-08-25T21:09:41.016 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86610.log 2024-08-25T21:09:41.016 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123033.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123033.log.gz 2024-08-25T21:09:41.017 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62978.log 2024-08-25T21:09:41.018 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86610.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86610.log.gz 2024-08-25T21:09:41.018 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109382.log 2024-08-25T21:09:41.019 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62978.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62978.log.gz 2024-08-25T21:09:41.019 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56193.log 2024-08-25T21:09:41.019 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109382.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109382.log.gz 2024-08-25T21:09:41.020 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88271.log 2024-08-25T21:09:41.020 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56193.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56193.log.gz 2024-08-25T21:09:41.020 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137612.log 2024-08-25T21:09:41.021 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88271.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88271.log.gz 2024-08-25T21:09:41.021 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115232.log 2024-08-25T21:09:41.022 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137612.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137612.log.gz 2024-08-25T21:09:41.022 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57546.log 2024-08-25T21:09:41.023 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115232.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115232.log.gz 2024-08-25T21:09:41.023 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57804.log 2024-08-25T21:09:41.024 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57546.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57546.log.gz 2024-08-25T21:09:41.024 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56435.log 2024-08-25T21:09:41.024 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57804.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57804.log.gz 2024-08-25T21:09:41.025 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133482.log 2024-08-25T21:09:41.026 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56435.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56435.log.gz 2024-08-25T21:09:41.026 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56093.log 2024-08-25T21:09:41.026 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133482.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133482.log.gz 2024-08-25T21:09:41.027 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85895.log 2024-08-25T21:09:41.028 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56093.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56093.log.gz 2024-08-25T21:09:41.028 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117872.log 2024-08-25T21:09:41.028 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85895.log.gz 2024-08-25T21:09:41.029 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94875.log 2024-08-25T21:09:41.029 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.117872.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117872.log.gz 2024-08-25T21:09:41.029 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90099.log 2024-08-25T21:09:41.030 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94875.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94875.log.gz 2024-08-25T21:09:41.030 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82869.log 2024-08-25T21:09:41.031 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90099.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90099.log.gz 2024-08-25T21:09:41.031 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107870.log 2024-08-25T21:09:41.032 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82869.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58338.log 2024-08-25T21:09:41.032 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82869.log.gz 2024-08-25T21:09:41.033 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107870.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107870.log.gz 2024-08-25T21:09:41.033 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.31138.log 2024-08-25T21:09:41.033 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58338.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58338.log.gz 2024-08-25T21:09:41.033 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141999.log 2024-08-25T21:09:41.034 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.31138.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.31138.log.gz -5 2024-08-25T21:09:41.034 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.50522.log 2024-08-25T21:09:41.035 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141999.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141999.log.gz 2024-08-25T21:09:41.035 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39669.log 2024-08-25T21:09:41.036 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50522.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50522.log.gz 2024-08-25T21:09:41.036 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106995.log 2024-08-25T21:09:41.037 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39669.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39669.log.gz 2024-08-25T21:09:41.037 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127078.log 2024-08-25T21:09:41.037 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106995.log.gz 2024-08-25T21:09:41.038 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128610.log 2024-08-25T21:09:41.038 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127078.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127078.log.gz 2024-08-25T21:09:41.039 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137945.log 2024-08-25T21:09:41.040 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128610.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128610.log.gz 2024-08-25T21:09:41.040 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63061.log 2024-08-25T21:09:41.040 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137945.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137945.log.gz 2024-08-25T21:09:41.041 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26713.log 2024-08-25T21:09:41.041 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63061.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63061.log.gz 2024-08-25T21:09:41.041 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130398.log 2024-08-25T21:09:41.042 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26713.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.33384.log 2024-08-25T21:09:41.042 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26713.log.gz 2024-08-25T21:09:41.042 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130398.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.130398.log.gz /var/log/ceph/cluster1-client.admin.127746.log 2024-08-25T21:09:41.043 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:41.043 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.33384.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78718.log 2024-08-25T21:09:41.043 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.33384.log.gz 2024-08-25T21:09:41.044 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127746.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127746.log.gz 2024-08-25T21:09:41.044 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120066.log 2024-08-25T21:09:41.045 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78718.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78718.log.gz 2024-08-25T21:09:41.045 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122135.log 2024-08-25T21:09:41.046 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120066.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120066.log.gz 2024-08-25T21:09:41.046 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124522.log 2024-08-25T21:09:41.047 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122135.log.gz 2024-08-25T21:09:41.047 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105083.log 2024-08-25T21:09:41.048 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124522.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124522.log.gz 2024-08-25T21:09:41.048 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42771.log 2024-08-25T21:09:41.049 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105083.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63396.log 2024-08-25T21:09:41.049 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105083.log.gz 2024-08-25T21:09:41.049 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.42771.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42771.log.gz 2024-08-25T21:09:41.050 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24432.log 2024-08-25T21:09:41.050 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63396.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63396.log.gz 2024-08-25T21:09:41.050 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90446.log 2024-08-25T21:09:41.051 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24432.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24432.log.gz 2024-08-25T21:09:41.051 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29321.log 2024-08-25T21:09:41.052 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90446.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90446.log.gz 2024-08-25T21:09:41.052 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65288.log 2024-08-25T21:09:41.053 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29321.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29321.log.gz -5 2024-08-25T21:09:41.053 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.71258.log 2024-08-25T21:09:41.054 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65288.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65288.log.gz 2024-08-25T21:09:41.054 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40129.log 2024-08-25T21:09:41.054 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71258.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71258.log.gz 2024-08-25T21:09:41.055 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132858.log 2024-08-25T21:09:41.055 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40129.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40129.log.gz 2024-08-25T21:09:41.055 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42823.log 2024-08-25T21:09:41.056 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132858.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132858.log.gz 2024-08-25T21:09:41.056 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52000.log 2024-08-25T21:09:41.057 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.42823.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42823.log.gz 2024-08-25T21:09:41.057 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58721.log 2024-08-25T21:09:41.058 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52000.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64950.log 2024-08-25T21:09:41.058 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52000.log.gz 2024-08-25T21:09:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.58721.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster2-client.admin.45826.log -- replaced with /var/log/ceph/cluster1-client.mirror.58721.log.gz 2024-08-25T21:09:41.059 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64950.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64950.log.gz 2024-08-25T21:09:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.21722.log 2024-08-25T21:09:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45826.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45826.log.gz 2024-08-25T21:09:41.061 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95796.log 2024-08-25T21:09:41.061 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.21722.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.21722.log.gz 2024-08-25T21:09:41.061 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94787.log 2024-08-25T21:09:41.062 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95796.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95796.log.gz 2024-08-25T21:09:41.062 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.22679.log 2024-08-25T21:09:41.063 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94787.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94787.log.gz 2024-08-25T21:09:41.063 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123621.log 2024-08-25T21:09:41.064 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.22679.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.22679.log.gz 2024-08-25T21:09:41.064 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41222.log 2024-08-25T21:09:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123621.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123621.log.gz 2024-08-25T21:09:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60543.log 2024-08-25T21:09:41.066 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41222.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41222.log.gz 2024-08-25T21:09:41.066 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123076.log 2024-08-25T21:09:41.067 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60543.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60543.log.gz 2024-08-25T21:09:41.067 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39165.log 2024-08-25T21:09:41.068 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123076.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.123076.log.gz -5 2024-08-25T21:09:41.068 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.76832.log 2024-08-25T21:09:41.068 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39165.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39165.log.gz 2024-08-25T21:09:41.069 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88793.log 2024-08-25T21:09:41.069 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76832.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76832.log.gz 2024-08-25T21:09:41.069 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31809.log 2024-08-25T21:09:41.070 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88793.log.gz 2024-08-25T21:09:41.070 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64155.log 2024-08-25T21:09:41.071 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31809.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31809.log.gz 2024-08-25T21:09:41.071 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113110.log 2024-08-25T21:09:41.072 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64155.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64155.log.gz 2024-08-25T21:09:41.072 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52202.log 2024-08-25T21:09:41.073 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113110.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113110.log.gz 2024-08-25T21:09:41.073 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116048.log 2024-08-25T21:09:41.074 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52202.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28393.log 2024-08-25T21:09:41.074 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52202.log.gz 2024-08-25T21:09:41.075 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.116048.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.116048.log.gz 2024-08-25T21:09:41.075 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29322.log 2024-08-25T21:09:41.075 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28393.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51066.log 2024-08-25T21:09:41.075 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28393.log.gz 2024-08-25T21:09:41.076 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29322.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29322.log.gz 2024-08-25T21:09:41.076 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56936.log 2024-08-25T21:09:41.077 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51066.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51066.log.gz 2024-08-25T21:09:41.077 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118173.log 2024-08-25T21:09:41.078 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.56936.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126216.log 2024-08-25T21:09:41.078 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56936.log.gz 2024-08-25T21:09:41.078 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.118173.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118173.log.gz 2024-08-25T21:09:41.079 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53044.log 2024-08-25T21:09:41.079 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126216.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126216.log.gz 2024-08-25T21:09:41.080 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109615.log 2024-08-25T21:09:41.080 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53044.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53044.log.gz 2024-08-25T21:09:41.081 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92861.log 2024-08-25T21:09:41.081 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109615.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95271.log 2024-08-25T21:09:41.082 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109615.log.gz 2024-08-25T21:09:41.082 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92861.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92861.log.gz 2024-08-25T21:09:41.082 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73199.log 2024-08-25T21:09:41.083 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95271.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95271.log.gz 2024-08-25T21:09:41.083 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64575.log 2024-08-25T21:09:41.084 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73199.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73199.log.gz 2024-08-25T21:09:41.084 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24286.log 2024-08-25T21:09:41.085 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64575.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64575.log.gz 2024-08-25T21:09:41.085 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61989.log 2024-08-25T21:09:41.086 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24286.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24286.log.gz 2024-08-25T21:09:41.086 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93457.log 2024-08-25T21:09:41.087 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61989.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61989.log.gz 2024-08-25T21:09:41.087 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110054.log 2024-08-25T21:09:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93457.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster2-client.admin.93457.log.gz -- 2024-08-25T21:09:41.092 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.101093.log 2024-08-25T21:09:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110054.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110054.log.gz 2024-08-25T21:09:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39207.log 2024-08-25T21:09:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101093.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101093.log.gz 2024-08-25T21:09:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108007.log 2024-08-25T21:09:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39207.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39207.log.gz 2024-08-25T21:09:41.092 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77613.log 2024-08-25T21:09:41.093 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108007.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56654.log 2024-08-25T21:09:41.093 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108007.log.gz 2024-08-25T21:09:41.093 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77613.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.77613.log.gz -- 2024-08-25T21:09:41.093 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.89261.log 2024-08-25T21:09:41.094 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56654.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56654.log.gz 2024-08-25T21:09:41.094 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44885.log 2024-08-25T21:09:41.094 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89261.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89261.log.gz 2024-08-25T21:09:41.094 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101425.log 2024-08-25T21:09:41.095 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44885.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44885.log.gz 2024-08-25T21:09:41.095 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41348.log 2024-08-25T21:09:41.095 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101425.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101425.log.gz 2024-08-25T21:09:41.096 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34256.log 2024-08-25T21:09:41.096 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41348.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41348.log.gz 2024-08-25T21:09:41.096 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73709.log 2024-08-25T21:09:41.097 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34256.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93730.log 2024-08-25T21:09:41.097 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34256.log.gz 2024-08-25T21:09:41.098 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73709.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73709.log.gz 2024-08-25T21:09:41.098 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124444.log 2024-08-25T21:09:41.098 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.93730.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93730.log.gz 2024-08-25T21:09:41.099 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90971.log 2024-08-25T21:09:41.100 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124444.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124444.log.gz 2024-08-25T21:09:41.100 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58084.log 2024-08-25T21:09:41.100 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90971.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90971.log.gz 2024-08-25T21:09:41.101 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49699.log 2024-08-25T21:09:41.101 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58084.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124145.log 2024-08-25T21:09:41.101 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58084.log.gz 2024-08-25T21:09:41.102 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49699.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49699.log.gz 2024-08-25T21:09:41.102 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85370.log 2024-08-25T21:09:41.103 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124145.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124145.log.gz 2024-08-25T21:09:41.103 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101195.log 2024-08-25T21:09:41.104 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85370.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85370.log.gz 2024-08-25T21:09:41.104 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51277.log 2024-08-25T21:09:41.104 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101195.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101195.log.gz 2024-08-25T21:09:41.105 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98867.log 2024-08-25T21:09:41.105 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51277.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34873.log 2024-08-25T21:09:41.106 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51277.log.gz 2024-08-25T21:09:41.106 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98867.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98867.log.gz 2024-08-25T21:09:41.106 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46634.log 2024-08-25T21:09:41.107 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34873.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34873.log.gz 2024-08-25T21:09:41.107 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94387.log 2024-08-25T21:09:41.108 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46634.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46634.log.gz 2024-08-25T21:09:41.108 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119411.log 2024-08-25T21:09:41.109 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94387.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94387.log.gz 2024-08-25T21:09:41.109 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49013.log 2024-08-25T21:09:41.110 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119411.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119411.log.gz 2024-08-25T21:09:41.110 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91446.log 2024-08-25T21:09:41.111 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49013.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49013.log.gz 2024-08-25T21:09:41.111 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70846.log 2024-08-25T21:09:41.112 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91446.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91446.log.gz 2024-08-25T21:09:41.112 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.142675.log 2024-08-25T21:09:41.113 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70846.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70846.log.gz 2024-08-25T21:09:41.113 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24958.log 2024-08-25T21:09:41.114 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.142675.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.142675.log.gz 2024-08-25T21:09:41.114 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41180.log 2024-08-25T21:09:41.115 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24958.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39835.log 2024-08-25T21:09:41.115 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24958.log.gz 2024-08-25T21:09:41.116 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41180.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41180.log.gz 2024-08-25T21:09:41.116 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28941.log 2024-08-25T21:09:41.116 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39835.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39835.log.gz 2024-08-25T21:09:41.117 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33911.log 2024-08-25T21:09:41.117 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28941.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28941.log.gz 2024-08-25T21:09:41.118 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28815.log 2024-08-25T21:09:41.118 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33911.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33911.log.gz 2024-08-25T21:09:41.118 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64994.log 2024-08-25T21:09:41.119 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28815.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28815.log.gz 2024-08-25T21:09:41.119 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99865.log 2024-08-25T21:09:41.120 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64994.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121628.log 2024-08-25T21:09:41.120 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64994.log.gz 2024-08-25T21:09:41.121 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99865.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99865.log.gz 2024-08-25T21:09:41.121 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46085.log 2024-08-25T21:09:41.121 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121628.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121628.log.gz 2024-08-25T21:09:41.121 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77761.log 2024-08-25T21:09:41.122 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.46085.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46085.log.gz 2024-08-25T21:09:41.122 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115078.log 2024-08-25T21:09:41.123 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77761.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77761.log.gz 2024-08-25T21:09:41.123 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100168.log 2024-08-25T21:09:41.124 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115078.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115078.log.gz 2024-08-25T21:09:41.124 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127208.log 2024-08-25T21:09:41.125 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100168.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100168.log.gz 2024-08-25T21:09:41.125 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70330.log 2024-08-25T21:09:41.126 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127208.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102382.log 2024-08-25T21:09:41.126 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127208.log.gz 2024-08-25T21:09:41.126 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70330.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70330.log.gz 2024-08-25T21:09:41.127 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24611.log 2024-08-25T21:09:41.127 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102382.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102382.log.gz 2024-08-25T21:09:41.128 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114563.log 2024-08-25T21:09:41.128 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24611.log.gz 2024-08-25T21:09:41.129 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26801.log 2024-08-25T21:09:41.129 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114563.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114563.log.gz 2024-08-25T21:09:41.129 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70391.log 2024-08-25T21:09:41.130 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26801.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26801.log.gz 2024-08-25T21:09:41.130 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138047.log 2024-08-25T21:09:41.131 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70391.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61338.log 2024-08-25T21:09:41.131 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70391.log.gz 2024-08-25T21:09:41.132 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138047.log.gz 2024-08-25T21:09:41.132 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85765.log 2024-08-25T21:09:41.132 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61338.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61338.log.gz 2024-08-25T21:09:41.133 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38468.log 2024-08-25T21:09:41.133 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85765.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85765.log.gz 2024-08-25T21:09:41.133 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75513.log 2024-08-25T21:09:41.134 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38468.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38468.log.gz 2024-08-25T21:09:41.134 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122301.log 2024-08-25T21:09:41.135 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75513.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38870.log 2024-08-25T21:09:41.135 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75513.log.gz 2024-08-25T21:09:41.136 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122301.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122301.log.gz 2024-08-25T21:09:41.136 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127974.log 2024-08-25T21:09:41.137 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38870.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38870.log.gz 2024-08-25T21:09:41.137 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92481.log 2024-08-25T21:09:41.138 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.127974.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127974.log.gz 2024-08-25T21:09:41.138 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134190.log 2024-08-25T21:09:41.139 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92481.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54302.log 2024-08-25T21:09:41.139 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92481.log.gz 2024-08-25T21:09:41.140 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134190.log.gz 2024-08-25T21:09:41.140 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74105.log 2024-08-25T21:09:41.140 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54302.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54302.log.gz 2024-08-25T21:09:41.140 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127384.log 2024-08-25T21:09:41.141 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74105.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.32311.log 2024-08-25T21:09:41.141 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74105.log.gz 2024-08-25T21:09:41.142 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127384.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127384.log.gz 2024-08-25T21:09:41.142 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58882.log 2024-08-25T21:09:41.143 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.32311.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32311.log.gz 2024-08-25T21:09:41.143 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76710.log 2024-08-25T21:09:41.144 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58882.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58882.log.gz 2024-08-25T21:09:41.144 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137470.log 2024-08-25T21:09:41.145 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76710.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76710.log.gz 2024-08-25T21:09:41.145 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91825.log 2024-08-25T21:09:41.146 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137470.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137470.log.gz 2024-08-25T21:09:41.146 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65624.log 2024-08-25T21:09:41.146 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91825.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91825.log.gz 2024-08-25T21:09:41.147 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69780.log 2024-08-25T21:09:41.147 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65624.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65624.log.gz 2024-08-25T21:09:41.148 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133422.log 2024-08-25T21:09:41.148 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.69780.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69780.log.gz 2024-08-25T21:09:41.148 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72694.log 2024-08-25T21:09:41.149 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133422.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117465.log 2024-08-25T21:09:41.149 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133422.log.gz 2024-08-25T21:09:41.150 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72694.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.72694.log.gz -5 2024-08-25T21:09:41.150 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.78898.log 2024-08-25T21:09:41.151 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117465.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117465.log.gz 2024-08-25T21:09:41.151 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125134.log 2024-08-25T21:09:41.152 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78898.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78898.log.gz 2024-08-25T21:09:41.152 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101851.log 2024-08-25T21:09:41.153 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125134.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125134.log.gz 2024-08-25T21:09:41.153 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91689.log 2024-08-25T21:09:41.153 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101851.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101851.log.gz 2024-08-25T21:09:41.154 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96553.log 2024-08-25T21:09:41.154 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91689.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91689.log.gz 2024-08-25T21:09:41.154 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43074.log 2024-08-25T21:09:41.155 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96553.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96553.log.gz 2024-08-25T21:09:41.155 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122255.log 2024-08-25T21:09:41.156 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43074.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43074.log.gz 2024-08-25T21:09:41.156 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101069.log 2024-08-25T21:09:41.157 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122255.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122255.log.gz 2024-08-25T21:09:41.157 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113694.log 2024-08-25T21:09:41.158 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101069.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101069.log.gz 2024-08-25T21:09:41.158 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59933.log 2024-08-25T21:09:41.159 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113694.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113694.log.gz 2024-08-25T21:09:41.159 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54673.log 2024-08-25T21:09:41.160 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59933.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59933.log.gz 2024-08-25T21:09:41.160 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28142.log 2024-08-25T21:09:41.160 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54673.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54673.log.gz 2024-08-25T21:09:41.161 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48760.log 2024-08-25T21:09:41.162 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28142.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28142.log.gz 2024-08-25T21:09:41.162 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49741.log 2024-08-25T21:09:41.162 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48760.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.90566.log 2024-08-25T21:09:41.163 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.48760.log.gz 2024-08-25T21:09:41.163 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49741.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49741.log.gz 2024-08-25T21:09:41.163 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97479.log 2024-08-25T21:09:41.164 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90566.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90566.log.gz 2024-08-25T21:09:41.164 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47033.log 2024-08-25T21:09:41.165 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97479.log.gz 2024-08-25T21:09:41.165 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53941.log 2024-08-25T21:09:41.165 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47033.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47033.log.gz 2024-08-25T21:09:41.166 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133842.log 2024-08-25T21:09:41.166 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53941.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53941.log.gz 2024-08-25T21:09:41.167 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119175.log 2024-08-25T21:09:41.168 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133842.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133842.log.gz 2024-08-25T21:09:41.168 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106200.log 2024-08-25T21:09:41.168 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119175.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119175.log.gz 2024-08-25T21:09:41.169 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93191.log 2024-08-25T21:09:41.169 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.106200.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106200.log.gz 2024-08-25T21:09:41.169 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65257.log 2024-08-25T21:09:41.170 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93191.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93191.log.gz 2024-08-25T21:09:41.170 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108565.log 2024-08-25T21:09:41.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65257.log.gz 2024-08-25T21:09:41.172 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63670.log 2024-08-25T21:09:41.172 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108565.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108565.log.gz 2024-08-25T21:09:41.172 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70883.log 2024-08-25T21:09:41.173 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63670.log.gz 2024-08-25T21:09:41.173 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.33107.log 2024-08-25T21:09:41.174 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70883.log.gz 2024-08-25T21:09:41.174 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55510.log 2024-08-25T21:09:41.175 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.33107.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.33107.log.gz 2024-08-25T21:09:41.175 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99053.log 2024-08-25T21:09:41.176 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55510.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55510.log.gz 2024-08-25T21:09:41.176 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78786.log 2024-08-25T21:09:41.177 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99053.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99053.log.gz 2024-08-25T21:09:41.177 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136838.log 2024-08-25T21:09:41.177 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78786.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78786.log.gz 2024-08-25T21:09:41.178 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118912.log 2024-08-25T21:09:41.178 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136838.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136838.log.gz 2024-08-25T21:09:41.179 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91691.log 2024-08-25T21:09:41.179 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118912.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118912.log.gz 2024-08-25T21:09:41.179 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.31086.log 2024-08-25T21:09:41.180 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91691.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91691.log.gz 2024-08-25T21:09:41.180 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.95339.log 2024-08-25T21:09:41.181 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.31086.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52286.log 2024-08-25T21:09:41.181 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.31086.log.gz 2024-08-25T21:09:41.182 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.95339.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50311.log 2024-08-25T21:09:41.182 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95339.log.gz 2024-08-25T21:09:41.183 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52286.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52286.log.gz 2024-08-25T21:09:41.183 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137403.log 2024-08-25T21:09:41.184 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50311.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50311.log.gz 2024-08-25T21:09:41.184 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97628.log 2024-08-25T21:09:41.185 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137403.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137403.log.gz 2024-08-25T21:09:41.185 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48259.log 2024-08-25T21:09:41.185 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97628.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97628.log.gz 2024-08-25T21:09:41.186 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113800.log 2024-08-25T21:09:41.186 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.48259.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48259.log.gz 2024-08-25T21:09:41.186 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89635.log 2024-08-25T21:09:41.187 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113800.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113800.log.gz 2024-08-25T21:09:41.187 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138356.log 2024-08-25T21:09:41.188 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89635.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89635.log.gz 2024-08-25T21:09:41.188 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138968.log 2024-08-25T21:09:41.189 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138356.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138356.log.gz 2024-08-25T21:09:41.189 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89826.log 2024-08-25T21:09:41.189 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138968.log: gzip 0.0% -5 --verbose -- -- replaced with /var/log/ceph/cluster1-client.admin.138968.log.gz /var/log/ceph/cluster1-client.mirror.52116.log 2024-08-25T21:09:41.190 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:41.190 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89826.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89826.log.gz 2024-08-25T21:09:41.190 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91358.log 2024-08-25T21:09:41.191 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52116.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52116.log.gz 2024-08-25T21:09:41.191 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33885.log 2024-08-25T21:09:41.192 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91358.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91358.log.gz 2024-08-25T21:09:41.192 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51614.log 2024-08-25T21:09:41.193 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33885.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33885.log.gz 2024-08-25T21:09:41.193 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123032.log 2024-08-25T21:09:41.193 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51614.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51614.log.gz 2024-08-25T21:09:41.194 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65876.log 2024-08-25T21:09:41.194 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123032.log.gz 2024-08-25T21:09:41.194 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32386.log 2024-08-25T21:09:41.195 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65876.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65876.log.gz 2024-08-25T21:09:41.195 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85601.log 2024-08-25T21:09:41.196 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32386.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32386.log.gz 2024-08-25T21:09:41.196 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31609.log 2024-08-25T21:09:41.197 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85601.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.85601.log.gz -5 2024-08-25T21:09:41.197 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.84745.log 2024-08-25T21:09:41.198 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31609.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38407.log 2024-08-25T21:09:41.198 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31609.log.gz 2024-08-25T21:09:41.198 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84745.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.43054.log 2024-08-25T21:09:41.199 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84745.log.gz 2024-08-25T21:09:41.199 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38407.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38407.log.gz 2024-08-25T21:09:41.199 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89889.log 2024-08-25T21:09:41.200 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.43054.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30536.log 2024-08-25T21:09:41.200 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.43054.log.gz 2024-08-25T21:09:41.201 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89889.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89889.log.gz 2024-08-25T21:09:41.201 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59866.log 2024-08-25T21:09:41.202 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30536.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99348.log 2024-08-25T21:09:41.202 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30536.log.gz 2024-08-25T21:09:41.202 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59866.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59866.log.gz 2024-08-25T21:09:41.202 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42769.log 2024-08-25T21:09:41.203 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99348.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99348.log.gz 2024-08-25T21:09:41.203 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97244.log 2024-08-25T21:09:41.204 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42769.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42769.log.gz 2024-08-25T21:09:41.204 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26082.log 2024-08-25T21:09:41.205 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97244.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97244.log.gz 2024-08-25T21:09:41.205 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54174.log 2024-08-25T21:09:41.206 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26082.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112562.log 2024-08-25T21:09:41.206 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26082.log.gz 2024-08-25T21:09:41.206 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54174.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54174.log.gz 2024-08-25T21:09:41.207 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39795.log 2024-08-25T21:09:41.207 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112562.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112562.log.gz 2024-08-25T21:09:41.207 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110092.log 2024-08-25T21:09:41.208 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39795.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39795.log.gz 2024-08-25T21:09:41.208 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78163.log 2024-08-25T21:09:41.209 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110092.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61864.log 2024-08-25T21:09:41.209 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110092.log.gz 2024-08-25T21:09:41.209 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78163.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78163.log.gz 2024-08-25T21:09:41.210 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131878.log 2024-08-25T21:09:41.210 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61864.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61674.log 2024-08-25T21:09:41.210 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61864.log.gz 2024-08-25T21:09:41.211 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131878.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98930.log 2024-08-25T21:09:41.211 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131878.log.gz 2024-08-25T21:09:41.212 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61674.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61674.log.gz 2024-08-25T21:09:41.212 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105899.log 2024-08-25T21:09:41.213 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98930.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98930.log.gz 2024-08-25T21:09:41.213 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96409.log 2024-08-25T21:09:41.214 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.105899.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.105899.log.gz 2024-08-25T21:09:41.214 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74970.log 2024-08-25T21:09:41.215 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96409.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96409.log.gz 2024-08-25T21:09:41.215 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100901.log 2024-08-25T21:09:41.215 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74970.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64658.log 2024-08-25T21:09:41.215 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74970.log.gz 2024-08-25T21:09:41.216 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100901.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81273.log 2024-08-25T21:09:41.216 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100901.log.gz 2024-08-25T21:09:41.217 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64658.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64658.log.gz 2024-08-25T21:09:41.217 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98888.log 2024-08-25T21:09:41.217 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.81273.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81273.log.gz 2024-08-25T21:09:41.218 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121627.log 2024-08-25T21:09:41.218 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98888.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98888.log.gz 2024-08-25T21:09:41.218 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95785.log 2024-08-25T21:09:41.219 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121627.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121627.log.gz 2024-08-25T21:09:41.219 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95448.log 2024-08-25T21:09:41.220 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95785.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52294.log 2024-08-25T21:09:41.220 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95785.log.gz 2024-08-25T21:09:41.220 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95448.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95448.log.gz 2024-08-25T21:09:41.221 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81574.log 2024-08-25T21:09:41.222 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52294.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52294.log.gz 2024-08-25T21:09:41.222 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64196.log 2024-08-25T21:09:41.222 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81574.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81574.log.gz 2024-08-25T21:09:41.223 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115276.log 2024-08-25T21:09:41.223 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64196.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64196.log.gz 2024-08-25T21:09:41.224 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122761.log 2024-08-25T21:09:41.224 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115276.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115276.log.gz 2024-08-25T21:09:41.224 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123521.log 2024-08-25T21:09:41.225 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122761.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122761.log.gz 2024-08-25T21:09:41.225 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44884.log 2024-08-25T21:09:41.226 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123521.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.123521.log.gz --verbose 2024-08-25T21:09:41.226 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster2-client.mirror.33008.log 2024-08-25T21:09:41.227 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44884.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44884.log.gz 2024-08-25T21:09:41.227 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.129450.log 2024-08-25T21:09:41.227 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.33008.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.33008.log.gz 2024-08-25T21:09:41.227 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38956.log 2024-08-25T21:09:41.228 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.129450.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.129450.log.gz 2024-08-25T21:09:41.228 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94970.log 2024-08-25T21:09:41.229 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38956.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38956.log.gz 2024-08-25T21:09:41.229 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80553.log 2024-08-25T21:09:41.230 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94970.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75703.log 2024-08-25T21:09:41.230 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94970.log.gz 2024-08-25T21:09:41.230 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80553.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80553.log.gz 2024-08-25T21:09:41.230 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25314.log 2024-08-25T21:09:41.231 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75703.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.mirror.75703.log.gz /var/log/ceph/cluster1-client.admin.67321.log 2024-08-25T21:09:41.231 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:41.232 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25314.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25314.log.gz 2024-08-25T21:09:41.232 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120893.log 2024-08-25T21:09:41.233 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67321.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67321.log.gz 2024-08-25T21:09:41.233 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45800.log 2024-08-25T21:09:41.234 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120893.log.gz 2024-08-25T21:09:41.234 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135997.log 2024-08-25T21:09:41.234 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45800.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.admin.45800.log.gz -5 2024-08-25T21:09:41.234 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.112248.log 2024-08-25T21:09:41.235 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135997.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135997.log.gz 2024-08-25T21:09:41.235 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50244.log 2024-08-25T21:09:41.236 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112248.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112248.log.gz 2024-08-25T21:09:41.236 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76629.log 2024-08-25T21:09:41.237 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50244.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50244.log.gz 2024-08-25T21:09:41.237 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84682.log 2024-08-25T21:09:41.238 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76629.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76629.log.gz 2024-08-25T21:09:41.238 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114930.log 2024-08-25T21:09:41.238 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84682.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84682.log.gz 2024-08-25T21:09:41.239 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137300.log 2024-08-25T21:09:41.239 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114930.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114930.log.gz 2024-08-25T21:09:41.239 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25482.log 2024-08-25T21:09:41.240 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137300.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137300.log.gz -5 2024-08-25T21:09:41.240 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.77971.log 2024-08-25T21:09:41.241 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25482.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25482.log.gz 2024-08-25T21:09:41.241 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136919.log 2024-08-25T21:09:41.241 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77971.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77971.log.gz 2024-08-25T21:09:41.241 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84035.log 2024-08-25T21:09:41.242 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136919.log.gz 2024-08-25T21:09:41.242 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24458.log 2024-08-25T21:09:41.243 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84035.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84035.log.gz 2024-08-25T21:09:41.243 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83749.log 2024-08-25T21:09:41.244 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24458.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24458.log.gz 2024-08-25T21:09:41.244 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118650.log 2024-08-25T21:09:41.245 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.83749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83749.log.gz 2024-08-25T21:09:41.245 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106241.log 2024-08-25T21:09:41.245 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118650.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118650.log.gz 2024-08-25T21:09:41.245 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86420.log 2024-08-25T21:09:41.246 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106241.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62074.log 2024-08-25T21:09:41.246 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106241.log.gz 2024-08-25T21:09:41.247 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86420.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113614.log 2024-08-25T21:09:41.247 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86420.log.gz 2024-08-25T21:09:41.248 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62074.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62074.log.gz 2024-08-25T21:09:41.248 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131279.log 2024-08-25T21:09:41.248 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113614.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113614.log.gz 2024-08-25T21:09:41.248 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44568.log 2024-08-25T21:09:41.249 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131279.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131279.log.gz 2024-08-25T21:09:41.249 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137614.log 2024-08-25T21:09:41.250 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44568.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44568.log.gz 2024-08-25T21:09:41.250 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76024.log 2024-08-25T21:09:41.250 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137614.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137614.log.gz 2024-08-25T21:09:41.251 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130950.log 2024-08-25T21:09:41.251 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76024.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76024.log.gz 2024-08-25T21:09:41.251 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88187.log 2024-08-25T21:09:41.252 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130950.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130950.log.gz 2024-08-25T21:09:41.252 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140719.log 2024-08-25T21:09:41.253 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88187.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88187.log.gz 2024-08-25T21:09:41.253 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131541.log 2024-08-25T21:09:41.254 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140719.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140719.log.gz 2024-08-25T21:09:41.254 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107089.log 2024-08-25T21:09:41.254 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131541.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131541.log.gz 2024-08-25T21:09:41.255 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69935.log 2024-08-25T21:09:41.255 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107089.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107089.log.gz 2024-08-25T21:09:41.255 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60966.log 2024-08-25T21:09:41.256 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.69935.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69935.log.gz 2024-08-25T21:09:41.256 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65413.log 2024-08-25T21:09:41.257 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60966.log.gz 2024-08-25T21:09:41.257 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23661.log 2024-08-25T21:09:41.258 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65413.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65413.log.gz 2024-08-25T21:09:41.258 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29463.log 2024-08-25T21:09:41.259 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23661.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28901.log 2024-08-25T21:09:41.259 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23661.log.gz 2024-08-25T21:09:41.260 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118974.log 2024-08-25T21:09:41.260 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29463.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29463.log.gz 2024-08-25T21:09:41.260 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28901.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28901.log.gz 2024-08-25T21:09:41.261 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82149.log 2024-08-25T21:09:41.261 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118974.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118974.log.gz 2024-08-25T21:09:41.262 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104502.log 2024-08-25T21:09:41.262 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82149.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82149.log.gz 2024-08-25T21:09:41.262 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34054.log 2024-08-25T21:09:41.263 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.104502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104502.log.gz 2024-08-25T21:09:41.263 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66295.log 2024-08-25T21:09:41.264 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34054.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34054.log.gz 2024-08-25T21:09:41.264 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45930.log 2024-08-25T21:09:41.265 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66295.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66295.log.gz 2024-08-25T21:09:41.265 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36031.log 2024-08-25T21:09:41.266 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.45930.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45930.log.gz 2024-08-25T21:09:41.266 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73622.log 2024-08-25T21:09:41.267 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36031.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68225.log 2024-08-25T21:09:41.267 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36031.log.gz 2024-08-25T21:09:41.267 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73622.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73622.log.gz 2024-08-25T21:09:41.268 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39247.log 2024-08-25T21:09:41.269 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68225.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56092.log 2024-08-25T21:09:41.269 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68225.log.gz 2024-08-25T21:09:41.269 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39247.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39247.log.gz 2024-08-25T21:09:41.269 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35253.log 2024-08-25T21:09:41.270 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56092.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56092.log.gz 2024-08-25T21:09:41.270 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40675.log 2024-08-25T21:09:41.271 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35253.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.35253.log.gz --verbose 2024-08-25T21:09:41.271 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.85577.log 2024-08-25T21:09:41.271 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40675.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40675.log.gz 2024-08-25T21:09:41.272 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91600.log 2024-08-25T21:09:41.272 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85577.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136123.log 2024-08-25T21:09:41.272 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85577.log.gz 2024-08-25T21:09:41.273 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91600.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91600.log.gz 2024-08-25T21:09:41.273 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29821.log 2024-08-25T21:09:41.274 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136123.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136123.log.gz 2024-08-25T21:09:41.274 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118355.log 2024-08-25T21:09:41.274 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29821.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29821.log.gz 2024-08-25T21:09:41.275 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29655.log 2024-08-25T21:09:41.275 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118355.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118355.log.gz 2024-08-25T21:09:41.275 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61800.log 2024-08-25T21:09:41.276 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29655.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74840.log 2024-08-25T21:09:41.276 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29655.log.gz 2024-08-25T21:09:41.277 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61800.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61800.log.gz 2024-08-25T21:09:41.277 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95538.log 2024-08-25T21:09:41.278 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74840.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74840.log.gz 2024-08-25T21:09:41.278 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91688.log 2024-08-25T21:09:41.278 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95538.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95538.log.gz 2024-08-25T21:09:41.278 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91776.log 2024-08-25T21:09:41.279 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91688.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91688.log.gz 2024-08-25T21:09:41.279 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61528.log 2024-08-25T21:09:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91776.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91776.log.gz 2024-08-25T21:09:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76103.log 2024-08-25T21:09:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61528.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43814.log 2024-08-25T21:09:41.281 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61528.log.gz 2024-08-25T21:09:41.281 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76103.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76103.log.gz 2024-08-25T21:09:41.281 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111742.log 2024-08-25T21:09:41.282 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43814.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43814.log.gz 2024-08-25T21:09:41.282 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26671.log 2024-08-25T21:09:41.283 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111742.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111742.log.gz 2024-08-25T21:09:41.283 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124674.log 2024-08-25T21:09:41.283 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26671.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26671.log.gz 2024-08-25T21:09:41.284 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141832.log 2024-08-25T21:09:41.285 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124674.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124674.log.gz 2024-08-25T21:09:41.285 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135432.log 2024-08-25T21:09:41.285 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141832.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94346.log 2024-08-25T21:09:41.286 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141832.log.gz 2024-08-25T21:09:41.286 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135432.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135432.log.gz 2024-08-25T21:09:41.286 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76102.log 2024-08-25T21:09:41.287 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94346.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94346.log.gz 2024-08-25T21:09:41.287 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122510.log 2024-08-25T21:09:41.288 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76102.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76102.log.gz 2024-08-25T21:09:41.288 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50034.log 2024-08-25T21:09:41.288 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122510.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122510.log.gz 2024-08-25T21:09:41.289 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73454.log 2024-08-25T21:09:41.289 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50034.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50034.log.gz 2024-08-25T21:09:41.289 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87912.log 2024-08-25T21:09:41.290 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73454.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73454.log.gz 2024-08-25T21:09:41.290 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32095.log 2024-08-25T21:09:41.291 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87912.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124634.log 2024-08-25T21:09:41.291 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87912.log.gz 2024-08-25T21:09:41.297 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32095.log.gz 2024-08-25T21:09:41.297 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124024.log 2024-08-25T21:09:41.297 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124634.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110797.log 0.0% 2024-08-25T21:09:41.297 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.124634.log.gz 2024-08-25T21:09:41.297 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124024.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124024.log.gz 2024-08-25T21:09:41.297 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78688.log 2024-08-25T21:09:41.297 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110797.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110797.log.gz 2024-08-25T21:09:41.298 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117426.log 2024-08-25T21:09:41.298 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78688.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78688.log.gz 2024-08-25T21:09:41.298 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65580.log 2024-08-25T21:09:41.298 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117426.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117426.log.gz 2024-08-25T21:09:41.298 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42358.log 2024-08-25T21:09:41.298 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65580.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65580.log.gz 2024-08-25T21:09:41.298 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118267.log 2024-08-25T21:09:41.298 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42358.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42358.log.gz 2024-08-25T21:09:41.298 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86926.log 2024-08-25T21:09:41.300 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.118267.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118267.log.gz 2024-08-25T21:09:41.300 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53967.log 2024-08-25T21:09:41.300 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86926.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86926.log.gz 2024-08-25T21:09:41.300 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83337.log 2024-08-25T21:09:41.300 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53967.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53967.log.gz 2024-08-25T21:09:41.300 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111384.log 2024-08-25T21:09:41.300 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83337.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83337.log.gz 2024-08-25T21:09:41.300 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72801.log 2024-08-25T21:09:41.301 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111384.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111384.log.gz 2024-08-25T21:09:41.301 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117110.log 2024-08-25T21:09:41.301 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72801.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72801.log.gz 2024-08-25T21:09:41.302 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41893.log 2024-08-25T21:09:41.303 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.117110.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117110.log.gz 2024-08-25T21:09:41.303 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49014.log 2024-08-25T21:09:41.303 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41893.log.gz 2024-08-25T21:09:41.304 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133303.log 2024-08-25T21:09:41.304 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49014.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49014.log.gz 2024-08-25T21:09:41.304 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54785.log 2024-08-25T21:09:41.305 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133303.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133303.log.gz 2024-08-25T21:09:41.305 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92483.log 2024-08-25T21:09:41.306 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54785.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54785.log.gz 2024-08-25T21:09:41.306 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74282.log 2024-08-25T21:09:41.307 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92483.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92483.log.gz 2024-08-25T21:09:41.307 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90355.log 2024-08-25T21:09:41.307 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74282.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74282.log.gz 2024-08-25T21:09:41.308 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132149.log 2024-08-25T21:09:41.308 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90355.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90355.log.gz 2024-08-25T21:09:41.308 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49450.log 2024-08-25T21:09:41.309 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132149.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132149.log.gz 2024-08-25T21:09:41.309 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97193.log 2024-08-25T21:09:41.310 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49450.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49450.log.gz 2024-08-25T21:09:41.310 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114299.log 2024-08-25T21:09:41.310 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97193.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97193.log.gz 2024-08-25T21:09:41.310 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119585.log 2024-08-25T21:09:41.311 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114299.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114299.log.gz 2024-08-25T21:09:41.311 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44034.log 2024-08-25T21:09:41.312 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119585.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119585.log.gz 2024-08-25T21:09:41.312 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101326.log 2024-08-25T21:09:41.313 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44034.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44034.log.gz 2024-08-25T21:09:41.313 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22942.log 2024-08-25T21:09:41.313 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101326.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101326.log.gz 2024-08-25T21:09:41.314 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141131.log 2024-08-25T21:09:41.314 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22942.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22942.log.gz 2024-08-25T21:09:41.314 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94004.log 2024-08-25T21:09:41.315 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.108095.log 2024-08-25T21:09:41.316 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141131.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141131.log.gz 2024-08-25T21:09:41.316 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94004.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94004.log.gz 2024-08-25T21:09:41.316 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.142717.log 2024-08-25T21:09:41.316 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.108095.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63440.log 2024-08-25T21:09:41.317 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.108095.log.gz 2024-08-25T21:09:41.317 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.142717.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.142717.log.gz 2024-08-25T21:09:41.318 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31785.log 2024-08-25T21:09:41.319 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63440.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63440.log.gz 2024-08-25T21:09:41.319 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142174.log 2024-08-25T21:09:41.319 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31785.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31785.log.gz 2024-08-25T21:09:41.319 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106448.log 2024-08-25T21:09:41.320 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.142174.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.142174.log.gz -5 2024-08-25T21:09:41.320 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.55620.log 2024-08-25T21:09:41.321 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.106448.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106448.log.gz 2024-08-25T21:09:41.321 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36939.log 2024-08-25T21:09:41.321 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55620.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55620.log.gz 2024-08-25T21:09:41.322 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100736.log 2024-08-25T21:09:41.322 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36939.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97420.log 2024-08-25T21:09:41.323 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36939.log.gz 2024-08-25T21:09:41.323 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100736.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100736.log.gz 2024-08-25T21:09:41.323 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79793.log 2024-08-25T21:09:41.324 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97420.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97420.log.gzgzip 2024-08-25T21:09:41.324 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120226.log 2024-08-25T21:09:41.325 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79793.log.gz 2024-08-25T21:09:41.325 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88620.log 2024-08-25T21:09:41.325 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120226.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136270.log 2024-08-25T21:09:41.326 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120226.log.gz 2024-08-25T21:09:41.326 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88620.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88620.log.gz 2024-08-25T21:09:41.326 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119364.log 2024-08-25T21:09:41.327 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136270.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136270.log.gz 2024-08-25T21:09:41.327 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27553.log 2024-08-25T21:09:41.328 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119364.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119364.log.gz 2024-08-25T21:09:41.328 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98466.log 2024-08-25T21:09:41.329 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27553.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27553.log.gz 2024-08-25T21:09:41.329 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42847.log 2024-08-25T21:09:41.330 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98466.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35178.log 2024-08-25T21:09:41.330 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98466.log.gz 2024-08-25T21:09:41.330 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.42847.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42847.log.gz 2024-08-25T21:09:41.330 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.142843.log 2024-08-25T21:09:41.331 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35178.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35178.log.gz 2024-08-25T21:09:41.331 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97419.log 2024-08-25T21:09:41.332 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.142843.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36003.log 2024-08-25T21:09:41.332 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.142843.log.gz 2024-08-25T21:09:41.333 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97419.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97419.log.gz 2024-08-25T21:09:41.333 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42229.log 2024-08-25T21:09:41.333 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36003.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125871.log 2024-08-25T21:09:41.333 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36003.log.gz 2024-08-25T21:09:41.334 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125284.log 2024-08-25T21:09:41.335 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42229.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42229.log.gz 2024-08-25T21:09:41.335 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125871.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86274.log 2024-08-25T21:09:41.335 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125871.log.gz 2024-08-25T21:09:41.336 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125284.log.gz 2024-08-25T21:09:41.336 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27783.log 2024-08-25T21:09:41.337 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86274.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86274.log.gz 2024-08-25T21:09:41.337 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24395.log 2024-08-25T21:09:41.338 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27783.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27783.log.gz 2024-08-25T21:09:41.338 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98174.log 2024-08-25T21:09:41.338 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24395.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24395.log.gz 2024-08-25T21:09:41.338 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126178.log 2024-08-25T21:09:41.339 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98174.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98174.log.gz 2024-08-25T21:09:41.339 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59511.log 2024-08-25T21:09:41.340 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126178.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126178.log.gz 2024-08-25T21:09:41.340 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115167.log 2024-08-25T21:09:41.341 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59511.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.59511.log.gz 2024-08-25T21:09:41.341 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86188.log 2024-08-25T21:09:41.342 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115167.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.admin.115167.log.gz -- 2024-08-25T21:09:41.342 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.138243.log 2024-08-25T21:09:41.342 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86188.log.gz 2024-08-25T21:09:41.343 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107175.log 2024-08-25T21:09:41.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.138243.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.138243.log.gz 2024-08-25T21:09:41.344 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41745.log 2024-08-25T21:09:41.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.107175.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107175.log.gz -5 2024-08-25T21:09:41.344 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.109821.log 2024-08-25T21:09:41.345 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23397.log 2024-08-25T21:09:41.346 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41745.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41745.log.gz 2024-08-25T21:09:41.346 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109821.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109821.log.gz 2024-08-25T21:09:41.346 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25211.log 2024-08-25T21:09:41.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23397.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23397.log.gz 2024-08-25T21:09:41.347 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.142361.log 2024-08-25T21:09:41.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25211.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25211.log.gz 2024-08-25T21:09:41.347 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78097.log 2024-08-25T21:09:41.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.142361.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.142361.log.gz 2024-08-25T21:09:41.348 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99226.log 2024-08-25T21:09:41.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78097.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109230.log 2024-08-25T21:09:41.349 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78097.log.gz 2024-08-25T21:09:41.349 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99226.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99226.log.gz 2024-08-25T21:09:41.350 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73095.log 2024-08-25T21:09:41.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109230.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109230.log.gz 2024-08-25T21:09:41.350 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121422.log 2024-08-25T21:09:41.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73095.log.gzgzip 2024-08-25T21:09:41.351 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90204.log 2024-08-25T21:09:41.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121422.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121422.log.gz 2024-08-25T21:09:41.352 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32744.log 2024-08-25T21:09:41.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90204.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90204.log.gz 2024-08-25T21:09:41.352 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99014.log 2024-08-25T21:09:41.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32744.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32744.log.gz 2024-08-25T21:09:41.353 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77891.log 2024-08-25T21:09:41.354 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99014.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99014.log.gz 2024-08-25T21:09:41.354 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92334.log 2024-08-25T21:09:41.355 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77891.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77891.log.gz 2024-08-25T21:09:41.355 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39459.log 2024-08-25T21:09:41.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92334.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92334.log.gz 2024-08-25T21:09:41.356 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77111.log 2024-08-25T21:09:41.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39459.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39459.log.gz 2024-08-25T21:09:41.357 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130362.log 2024-08-25T21:09:41.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77111.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77111.log.gz 2024-08-25T21:09:41.357 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104078.log 2024-08-25T21:09:41.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130362.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130362.log.gz 2024-08-25T21:09:41.358 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117010.log 2024-08-25T21:09:41.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104078.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104078.log.gz 2024-08-25T21:09:41.359 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38261.log 2024-08-25T21:09:41.360 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117010.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105204.log 2024-08-25T21:09:41.360 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117010.log.gz 2024-08-25T21:09:41.360 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38261.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38261.log.gz 2024-08-25T21:09:41.360 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113884.log 2024-08-25T21:09:41.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105204.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105204.log.gz 2024-08-25T21:09:41.361 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111385.log 2024-08-25T21:09:41.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113884.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113884.log.gz 2024-08-25T21:09:41.362 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.138213.log 2024-08-25T21:09:41.363 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111385.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111385.log.gz 2024-08-25T21:09:41.363 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66487.log 2024-08-25T21:09:41.363 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.138213.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.138213.log.gz 2024-08-25T21:09:41.364 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104079.log 2024-08-25T21:09:41.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66487.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66487.log.gz 2024-08-25T21:09:41.364 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112938.log 2024-08-25T21:09:41.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104079.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104079.log.gz 2024-08-25T21:09:41.365 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24792.log 2024-08-25T21:09:41.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112938.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112938.log.gz 2024-08-25T21:09:41.366 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58540.log 2024-08-25T21:09:41.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24792.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24792.log.gz 2024-08-25T21:09:41.367 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116783.log 2024-08-25T21:09:41.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58540.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58540.log.gz 2024-08-25T21:09:41.368 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133438.log 2024-08-25T21:09:41.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116783.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116783.log.gz 2024-08-25T21:09:41.368 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52748.log 2024-08-25T21:09:41.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133438.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133438.log.gz 2024-08-25T21:09:41.369 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100437.log 2024-08-25T21:09:41.370 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52748.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52748.log.gz 2024-08-25T21:09:41.370 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106469.log 2024-08-25T21:09:41.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100437.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100437.log.gz 2024-08-25T21:09:41.371 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40423.log 2024-08-25T21:09:41.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106469.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55430.log 2024-08-25T21:09:41.372 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106469.log.gz 2024-08-25T21:09:41.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40423.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.40423.log.gz --verbose 2024-08-25T21:09:41.372 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.23917.log 2024-08-25T21:09:41.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55430.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22310.log 2024-08-25T21:09:41.373 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55430.log.gz 2024-08-25T21:09:41.374 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23917.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23917.log.gz 2024-08-25T21:09:41.374 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135874.log 2024-08-25T21:09:41.375 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22310.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.131313.log 2024-08-25T21:09:41.375 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22310.log.gz 2024-08-25T21:09:41.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135874.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42019.log 2024-08-25T21:09:41.376 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135874.log.gz 2024-08-25T21:09:41.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.131313.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.131313.log.gz 2024-08-25T21:09:41.376 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42990.log 2024-08-25T21:09:41.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42019.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.42019.log.gz 2024-08-25T21:09:41.377 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87578.log 2024-08-25T21:09:41.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42990.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42990.log.gz 2024-08-25T21:09:41.378 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45904.log 2024-08-25T21:09:41.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87578.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67253.log 2024-08-25T21:09:41.378 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87578.log.gz 2024-08-25T21:09:41.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.45904.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45904.log.gz 2024-08-25T21:09:41.379 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109695.log 2024-08-25T21:09:41.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67253.log.gz 2024-08-25T21:09:41.380 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62557.log 2024-08-25T21:09:41.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109695.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109695.log.gz 2024-08-25T21:09:41.381 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132732.log 2024-08-25T21:09:41.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62557.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62557.log.gz 2024-08-25T21:09:41.382 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82719.log 2024-08-25T21:09:41.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132732.log.gz 2024-08-25T21:09:41.382 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97708.log 2024-08-25T21:09:41.383 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82719.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82719.log.gz 2024-08-25T21:09:41.383 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114509.log 2024-08-25T21:09:41.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97708.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97708.log.gz 2024-08-25T21:09:41.384 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63987.log 2024-08-25T21:09:41.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114509.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114509.log.gz 2024-08-25T21:09:41.385 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49074.log 2024-08-25T21:09:41.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63987.log.gz 2024-08-25T21:09:41.385 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118328.log 2024-08-25T21:09:41.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49074.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49074.log.gz 2024-08-25T21:09:41.386 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85266.log 2024-08-25T21:09:41.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118328.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118328.log.gz 2024-08-25T21:09:41.387 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32005.log 2024-08-25T21:09:41.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85266.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85266.log.gz 2024-08-25T21:09:41.388 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76546.log 2024-08-25T21:09:41.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32005.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32005.log.gz 2024-08-25T21:09:41.388 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45451.log 2024-08-25T21:09:41.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76546.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76546.log.gz 2024-08-25T21:09:41.389 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36271.log 2024-08-25T21:09:41.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45451.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30120.log 2024-08-25T21:09:41.390 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45451.log.gz 2024-08-25T21:09:41.391 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36271.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36271.log.gz 2024-08-25T21:09:41.391 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136836.log 2024-08-25T21:09:41.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30120.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30120.log.gz 2024-08-25T21:09:41.392 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91272.log 2024-08-25T21:09:41.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136836.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136836.log.gz 2024-08-25T21:09:41.392 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109108.log 2024-08-25T21:09:41.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91272.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72174.log 2024-08-25T21:09:41.393 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91272.log.gz 2024-08-25T21:09:41.394 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109108.log.gz 2024-08-25T21:09:41.394 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44386.log 2024-08-25T21:09:41.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72174.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72174.log.gz 2024-08-25T21:09:41.395 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37214.log 2024-08-25T21:09:41.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44386.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44386.log.gz 2024-08-25T21:09:41.395 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139889.log 2024-08-25T21:09:41.396 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37214.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.37214.log.gz 2024-08-25T21:09:41.396 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35291.log 2024-08-25T21:09:41.397 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139889.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139889.log.gz 2024-08-25T21:09:41.397 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136479.log 2024-08-25T21:09:41.398 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35291.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35291.log.gz 2024-08-25T21:09:41.398 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98235.log 2024-08-25T21:09:41.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136479.log.gz 2024-08-25T21:09:41.399 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24852.log 2024-08-25T21:09:41.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98235.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98235.log.gz 2024-08-25T21:09:41.399 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102527.log 2024-08-25T21:09:41.400 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24852.log.gz 2024-08-25T21:09:41.400 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60542.log 2024-08-25T21:09:41.401 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102527.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102527.log.gz 2024-08-25T21:09:41.401 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35724.log 2024-08-25T21:09:41.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60542.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60542.log.gz 2024-08-25T21:09:41.402 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105445.log 2024-08-25T21:09:41.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35724.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35724.log.gz 2024-08-25T21:09:41.403 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27176.log 2024-08-25T21:09:41.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105445.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105445.log.gz 2024-08-25T21:09:41.403 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71611.log 2024-08-25T21:09:41.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27176.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27176.log.gz 2024-08-25T21:09:41.404 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64406.log 2024-08-25T21:09:41.405 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.71611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71611.log.gz 2024-08-25T21:09:41.405 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137722.log 2024-08-25T21:09:41.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64406.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64406.log.gz 2024-08-25T21:09:41.406 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100417.log 2024-08-25T21:09:41.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137722.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137722.log.gz 2024-08-25T21:09:41.407 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39249.log 2024-08-25T21:09:41.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100417.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100417.log.gz 2024-08-25T21:09:41.408 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76879.log 2024-08-25T21:09:41.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39249.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110326.log 2024-08-25T21:09:41.409 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39249.log.gz 2024-08-25T21:09:41.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76879.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76879.log.gz 2024-08-25T21:09:41.409 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46702.log 2024-08-25T21:09:41.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110326.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110326.log.gz 2024-08-25T21:09:41.410 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29462.log 2024-08-25T21:09:41.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46702.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46702.log.gz 2024-08-25T21:09:41.411 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2.audit.log 2024-08-25T21:09:41.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29462.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.52368.log -- replaced with /var/log/ceph/cluster1-client.mirror.29462.log.gz 2024-08-25T21:09:41.411 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:41.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2.audit.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115101.log 2024-08-25T21:09:41.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52368.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52368.log.gz 2024-08-25T21:09:41.413 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52244.log 2024-08-25T21:09:41.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115101.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115101.log.gz 2024-08-25T21:09:41.414 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30210.log 2024-08-25T21:09:41.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52244.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52244.log.gz 2024-08-25T21:09:41.415 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98256.log 2024-08-25T21:09:41.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30210.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30210.log.gz 2024-08-25T21:09:41.415 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114987.log 2024-08-25T21:09:41.416 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98256.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98256.log.gz 2024-08-25T21:09:41.416 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63535.log 2024-08-25T21:09:41.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114987.log.gz 2024-08-25T21:09:41.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134402.log 2024-08-25T21:09:41.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63535.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63535.log.gz 2024-08-25T21:09:41.418 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84744.log 2024-08-25T21:09:41.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134402.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134402.log.gz 2024-08-25T21:09:41.418 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40929.log 2024-08-25T21:09:41.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84744.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84744.log.gz 2024-08-25T21:09:41.419 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73863.log 2024-08-25T21:09:41.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40929.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40929.log.gz 2024-08-25T21:09:41.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63774.log 2024-08-25T21:09:41.421 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134012.log 2024-08-25T21:09:41.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73863.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73863.log.gz 2024-08-25T21:09:41.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63774.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63774.log.gz 2024-08-25T21:09:41.422 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58171.log 2024-08-25T21:09:41.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134012.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134012.log.gz 2024-08-25T21:09:41.423 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98761.log 2024-08-25T21:09:41.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58171.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58171.log.gz 2024-08-25T21:09:41.423 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36855.log 2024-08-25T21:09:41.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98761.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98761.log.gz 2024-08-25T21:09:41.424 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115365.log 2024-08-25T21:09:41.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36855.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36855.log.gz 2024-08-25T21:09:41.425 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132355.log 2024-08-25T21:09:41.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115365.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115365.log.gz 2024-08-25T21:09:41.426 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99391.log 2024-08-25T21:09:41.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132355.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132355.log.gz 2024-08-25T21:09:41.427 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69988.log 2024-08-25T21:09:41.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99391.log.gz 2024-08-25T21:09:41.427 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46520.log 2024-08-25T21:09:41.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.69988.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69988.log.gz 2024-08-25T21:09:41.428 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85811.log 2024-08-25T21:09:41.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46520.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46520.log.gz 2024-08-25T21:09:41.429 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95032.log 2024-08-25T21:09:41.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85811.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85811.log.gz 2024-08-25T21:09:41.430 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64404.log 2024-08-25T21:09:41.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95032.log.gz 2024-08-25T21:09:41.431 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69801.log 2024-08-25T21:09:41.431 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64404.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64404.log.gz 2024-08-25T21:09:41.431 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135705.log 2024-08-25T21:09:41.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.69801.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.69801.log.gz 2024-08-25T21:09:41.432 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99665.log 2024-08-25T21:09:41.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135705.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135705.log.gz 2024-08-25T21:09:41.433 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53208.log 2024-08-25T21:09:41.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99665.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99665.log.gz 2024-08-25T21:09:41.434 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105832.log 2024-08-25T21:09:41.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53208.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.142969.log 2024-08-25T21:09:41.435 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53208.log.gz 2024-08-25T21:09:41.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.105832.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105832.log.gz 2024-08-25T21:09:41.436 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139275.log 2024-08-25T21:09:41.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.142969.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.142969.log.gz 2024-08-25T21:09:41.436 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96912.log 2024-08-25T21:09:41.437 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139275.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139275.log.gz 2024-08-25T21:09:41.437 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.128121.log 2024-08-25T21:09:41.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96912.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96912.log.gz 2024-08-25T21:09:41.438 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.22788.log 2024-08-25T21:09:41.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.128121.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.128121.log.gz 2024-08-25T21:09:41.439 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115032.log 2024-08-25T21:09:41.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.22788.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.22788.log.gz 2024-08-25T21:09:41.439 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134033.log 2024-08-25T21:09:41.440 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115032.log.gz 2024-08-25T21:09:41.440 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-mgr.x.log 2024-08-25T21:09:41.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52914.log 2024-08-25T21:09:41.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134033.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134033.log.gz 2024-08-25T21:09:41.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75828.log 2024-08-25T21:09:41.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52914.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52914.log.gz 2024-08-25T21:09:41.443 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44231.log 2024-08-25T21:09:41.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75828.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108982.log 2024-08-25T21:09:41.444 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75828.log.gz 2024-08-25T21:09:41.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.44231.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103969.log 2024-08-25T21:09:41.445 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44231.log.gz 2024-08-25T21:09:41.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108982.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128725.log 2024-08-25T21:09:41.445 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108982.log.gz 2024-08-25T21:09:41.446 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103969.log.gzgzip 2024-08-25T21:09:41.446 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135616.log 2024-08-25T21:09:41.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.128725.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128725.log.gz 2024-08-25T21:09:41.447 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41944.log 2024-08-25T21:09:41.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135616.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135616.log.gz 2024-08-25T21:09:41.448 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90367.log 2024-08-25T21:09:41.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41944.log.gz 2024-08-25T21:09:41.449 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129523.log 2024-08-25T21:09:41.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90367.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90743.log 2024-08-25T21:09:41.449 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90367.log.gz 2024-08-25T21:09:41.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129523.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129523.log.gz 2024-08-25T21:09:41.450 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36446.log 2024-08-25T21:09:41.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90743.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90743.log.gz 2024-08-25T21:09:41.451 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55421.log 2024-08-25T21:09:41.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36446.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36446.log.gz 2024-08-25T21:09:41.452 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113886.log 2024-08-25T21:09:41.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55421.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55421.log.gz 2024-08-25T21:09:41.452 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139769.log 2024-08-25T21:09:41.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113886.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113886.log.gz 2024-08-25T21:09:41.453 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56974.log 2024-08-25T21:09:41.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139769.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139769.log.gz 2024-08-25T21:09:41.454 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24482.log 2024-08-25T21:09:41.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56974.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56974.log.gz 2024-08-25T21:09:41.455 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120852.log 2024-08-25T21:09:41.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24482.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24482.log.gz 2024-08-25T21:09:41.455 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133234.log 2024-08-25T21:09:41.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120852.log.gz 2024-08-25T21:09:41.456 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130577.log 2024-08-25T21:09:41.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133234.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133234.log.gz 2024-08-25T21:09:41.457 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38702.log 2024-08-25T21:09:41.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.130577.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.67429.log 2024-08-25T21:09:41.458 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.130577.log.gz 2024-08-25T21:09:41.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38702.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38702.log.gz 2024-08-25T21:09:41.458 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109274.log 2024-08-25T21:09:41.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67429.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67429.log.gz 2024-08-25T21:09:41.459 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89823.log 2024-08-25T21:09:41.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109274.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118669.log 2024-08-25T21:09:41.460 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109274.log.gz 2024-08-25T21:09:41.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89823.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89823.log.gz 2024-08-25T21:09:41.461 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64952.log 2024-08-25T21:09:41.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118669.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118669.log.gz 2024-08-25T21:09:41.461 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30989.log 2024-08-25T21:09:41.462 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64952.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64952.log.gz 2024-08-25T21:09:41.462 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37945.log 2024-08-25T21:09:41.463 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.30989.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30989.log.gz 2024-08-25T21:09:41.463 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62658.log 2024-08-25T21:09:41.463 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37945.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37945.log.gz 2024-08-25T21:09:41.464 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96239.log 2024-08-25T21:09:41.464 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62658.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62658.log.gz 2024-08-25T21:09:41.464 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42917.log 2024-08-25T21:09:41.465 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96239.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50622.log 2024-08-25T21:09:41.465 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96239.log.gz 2024-08-25T21:09:41.466 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.42917.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116782.log 2024-08-25T21:09:41.466 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42917.log.gz 2024-08-25T21:09:41.466 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50622.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.50622.log.gz 2024-08-25T21:09:41.466 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.87271.log 2024-08-25T21:09:41.467 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116782.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116782.log.gz 2024-08-25T21:09:41.467 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109086.log 2024-08-25T21:09:41.468 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87271.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87271.log.gz 2024-08-25T21:09:41.468 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64783.log 2024-08-25T21:09:41.469 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109086.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.109086.log.gz --verbose 2024-08-25T21:09:41.469 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.admin.103708.log 2024-08-25T21:09:41.469 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64783.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64783.log.gz 2024-08-25T21:09:41.470 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67966.log 2024-08-25T21:09:41.470 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103708.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103708.log.gz 2024-08-25T21:09:41.470 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58285.log 2024-08-25T21:09:41.471 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67966.log.gz 2024-08-25T21:09:41.471 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79792.log 2024-08-25T21:09:41.472 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58285.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58285.log.gz 2024-08-25T21:09:41.472 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64362.log 2024-08-25T21:09:41.473 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79792.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79792.log.gz 2024-08-25T21:09:41.473 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43140.log 2024-08-25T21:09:41.473 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64362.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.64362.log.gz --verbose 2024-08-25T21:09:41.474 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.50102.log 2024-08-25T21:09:41.474 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43140.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139990.log 2024-08-25T21:09:41.474 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43140.log.gz 2024-08-25T21:09:41.475 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50102.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88961.log 2024-08-25T21:09:41.475 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50102.log.gz 2024-08-25T21:09:41.476 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.139990.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.admin.139990.log.gz 2024-08-25T21:09:41.476 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.55399.log 2024-08-25T21:09:41.477 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88961.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48170.log 2024-08-25T21:09:41.477 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88961.log.gz 2024-08-25T21:09:41.477 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55399.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55399.log.gz 2024-08-25T21:09:41.478 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37273.log 2024-08-25T21:09:41.478 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48170.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48170.log.gz 2024-08-25T21:09:41.478 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30494.log 2024-08-25T21:09:41.479 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37273.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37273.log.gz 2024-08-25T21:09:41.479 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115650.log 2024-08-25T21:09:41.480 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30494.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.30494.log.gz 2024-08-25T21:09:41.480 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111344.log 2024-08-25T21:09:41.481 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115650.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115650.log.gz 2024-08-25T21:09:41.481 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66558.log 2024-08-25T21:09:41.481 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49344.log 2024-08-25T21:09:41.482 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111344.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111344.log.gz 2024-08-25T21:09:41.482 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66558.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66558.log.gz 2024-08-25T21:09:41.482 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133611.log 2024-08-25T21:09:41.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49344.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49344.log.gz 2024-08-25T21:09:41.483 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99433.log 2024-08-25T21:09:41.484 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133611.log.gz 2024-08-25T21:09:41.484 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101469.log 2024-08-25T21:09:41.485 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99433.log.gz 2024-08-25T21:09:41.485 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120707.log 2024-08-25T21:09:41.486 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101469.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113944.log 2024-08-25T21:09:41.486 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101469.log.gz 2024-08-25T21:09:41.486 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120707.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120707.log.gz 2024-08-25T21:09:41.487 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108416.log 2024-08-25T21:09:41.487 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113944.log.gz 2024-08-25T21:09:41.487 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49891.log 2024-08-25T21:09:41.488 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108416.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108416.log.gz 2024-08-25T21:09:41.488 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117945.log 2024-08-25T21:09:41.489 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49891.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49891.log.gz 2024-08-25T21:09:41.489 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134652.log 2024-08-25T21:09:41.490 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.117945.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117945.log.gz 2024-08-25T21:09:41.490 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86885.log 2024-08-25T21:09:41.490 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134652.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134652.log.gzgzip 2024-08-25T21:09:41.490 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.55952.log 2024-08-25T21:09:41.491 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86885.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86885.log.gz 2024-08-25T21:09:41.491 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.138279.log 2024-08-25T21:09:41.492 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55952.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.55952.log.gz 2024-08-25T21:09:41.492 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.115900.log 2024-08-25T21:09:41.493 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.138279.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.138279.log.gz 2024-08-25T21:09:41.493 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69576.log 2024-08-25T21:09:41.494 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.115900.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.115900.log.gz 2024-08-25T21:09:41.494 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31874.log 2024-08-25T21:09:41.494 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69576.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69576.log.gz 2024-08-25T21:09:41.495 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78902.log 2024-08-25T21:09:41.501 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25937.log 2024-08-25T21:09:41.501 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31874.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31874.log.gz 2024-08-25T21:09:41.501 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78902.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78902.log.gz 2024-08-25T21:09:41.501 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35965.log 2024-08-25T21:09:41.501 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25937.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25937.log.gz 2024-08-25T21:09:41.501 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97629.log 2024-08-25T21:09:41.501 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35965.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35965.log.gz 2024-08-25T21:09:41.501 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142268.log 2024-08-25T21:09:41.501 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97629.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97629.log.gz 2024-08-25T21:09:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29552.log 2024-08-25T21:09:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.142268.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142268.log.gz 2024-08-25T21:09:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57765.log 2024-08-25T21:09:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29552.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29552.log.gz 2024-08-25T21:09:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64320.log 2024-08-25T21:09:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57765.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57765.log.gz 2024-08-25T21:09:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136417.log 2024-08-25T21:09:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64320.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64320.log.gz 2024-08-25T21:09:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124460.log 2024-08-25T21:09:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136417.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136417.log.gz 2024-08-25T21:09:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61633.log 2024-08-25T21:09:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124460.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124460.log.gz 2024-08-25T21:09:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37904.log 2024-08-25T21:09:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61633.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61633.log.gz 2024-08-25T21:09:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22899.log 2024-08-25T21:09:41.505 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37904.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37904.log.gz 2024-08-25T21:09:41.506 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67296.log 2024-08-25T21:09:41.506 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22899.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22899.log.gz 2024-08-25T21:09:41.506 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83900.log 2024-08-25T21:09:41.506 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67296.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67296.log.gz 2024-08-25T21:09:41.506 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120390.log 2024-08-25T21:09:41.507 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.83900.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83900.log.gz 2024-08-25T21:09:41.507 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51404.log 2024-08-25T21:09:41.508 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120390.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120390.log.gz 2024-08-25T21:09:41.508 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38513.log 2024-08-25T21:09:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51404.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51404.log.gz 2024-08-25T21:09:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102144.log 2024-08-25T21:09:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38513.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38513.log.gz 2024-08-25T21:09:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68910.log 2024-08-25T21:09:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102144.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102144.log.gz 2024-08-25T21:09:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23098.log 2024-08-25T21:09:41.511 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.68910.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.68910.log.gz 2024-08-25T21:09:41.511 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100107.log 2024-08-25T21:09:41.512 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23098.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23098.log.gz 2024-08-25T21:09:41.512 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135179.log 2024-08-25T21:09:41.512 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100107.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100107.log.gz 2024-08-25T21:09:41.513 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127479.log 2024-08-25T21:09:41.513 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135179.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135179.log.gz 2024-08-25T21:09:41.513 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77003.log 2024-08-25T21:09:41.514 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127479.log.gz 2024-08-25T21:09:41.514 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79018.log 2024-08-25T21:09:41.515 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77003.log.gz 2024-08-25T21:09:41.515 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101027.log 2024-08-25T21:09:41.516 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79018.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79018.log.gz 2024-08-25T21:09:41.516 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53630.log 2024-08-25T21:09:41.516 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101027.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101027.log.gz 2024-08-25T21:09:41.517 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136210.log 2024-08-25T21:09:41.517 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53630.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53630.log.gz 2024-08-25T21:09:41.517 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38008.log 2024-08-25T21:09:41.518 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136210.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136210.log.gz 2024-08-25T21:09:41.518 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42331.log 2024-08-25T21:09:41.519 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38008.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38008.log.gz 2024-08-25T21:09:41.519 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131880.log 2024-08-25T21:09:41.520 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42331.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42331.log.gz 2024-08-25T21:09:41.520 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79878.log 2024-08-25T21:09:41.520 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131880.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131880.log.gz 2024-08-25T21:09:41.520 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58985.log 2024-08-25T21:09:41.521 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79878.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79878.log.gz 2024-08-25T21:09:41.521 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85372.log 2024-08-25T21:09:41.522 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.58985.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58985.log.gz 2024-08-25T21:09:41.522 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96097.log 2024-08-25T21:09:41.523 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85372.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85372.log.gz 2024-08-25T21:09:41.523 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41853.log 2024-08-25T21:09:41.524 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96097.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96097.log.gz 2024-08-25T21:09:41.524 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90388.log 2024-08-25T21:09:41.524 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41853.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52474.log 2024-08-25T21:09:41.525 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41853.log.gz 2024-08-25T21:09:41.525 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90388.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90388.log.gz 2024-08-25T21:09:41.525 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92158.log 2024-08-25T21:09:41.526 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52474.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52474.log.gz 2024-08-25T21:09:41.526 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43463.log 2024-08-25T21:09:41.527 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92158.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92158.log.gz 2024-08-25T21:09:41.527 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100254.log 2024-08-25T21:09:41.527 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43463.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43463.log.gz 2024-08-25T21:09:41.528 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36182.log 2024-08-25T21:09:41.528 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100254.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100254.log.gz 2024-08-25T21:09:41.528 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92965.log 2024-08-25T21:09:41.529 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36182.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36182.log.gzgzip 2024-08-25T21:09:41.529 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48804.log 2024-08-25T21:09:41.530 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92965.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92965.log.gz 2024-08-25T21:09:41.530 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132562.log 2024-08-25T21:09:41.531 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48804.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48804.log.gz 2024-08-25T21:09:41.531 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118948.log 2024-08-25T21:09:41.531 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132562.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132562.log.gz 2024-08-25T21:09:41.532 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71875.log 2024-08-25T21:09:41.532 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118948.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118948.log.gz 2024-08-25T21:09:41.532 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81239.log 2024-08-25T21:09:41.533 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71875.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71875.log.gz 2024-08-25T21:09:41.533 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112620.log 2024-08-25T21:09:41.534 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.81239.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81239.log.gz 2024-08-25T21:09:41.534 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94186.log 2024-08-25T21:09:41.535 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112620.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112620.log.gz 2024-08-25T21:09:41.535 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65884.log 2024-08-25T21:09:41.535 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94186.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94186.log.gz 2024-08-25T21:09:41.536 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115366.log 2024-08-25T21:09:41.536 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65884.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65884.log.gz 2024-08-25T21:09:41.537 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106697.log 2024-08-25T21:09:41.537 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115366.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115366.log.gz 2024-08-25T21:09:41.538 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27222.log 2024-08-25T21:09:41.538 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106697.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106697.log.gz 2024-08-25T21:09:41.538 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135346.log 2024-08-25T21:09:41.539 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27222.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116138.log 2024-08-25T21:09:41.539 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27222.log.gz 2024-08-25T21:09:41.540 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135346.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135346.log.gz 2024-08-25T21:09:41.540 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141356.log 2024-08-25T21:09:41.541 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116138.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116138.log.gz 2024-08-25T21:09:41.541 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86653.log 2024-08-25T21:09:41.541 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.141356.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141356.log.gz 2024-08-25T21:09:41.542 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86131.log 2024-08-25T21:09:41.542 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86653.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86653.log.gz 2024-08-25T21:09:41.542 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60816.log 2024-08-25T21:09:41.543 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86131.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85749.log 2024-08-25T21:09:41.543 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86131.log.gz 2024-08-25T21:09:41.544 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60816.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60816.log.gz 2024-08-25T21:09:41.544 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123098.log 2024-08-25T21:09:41.545 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85749.log.gz 2024-08-25T21:09:41.545 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101101.log 2024-08-25T21:09:41.546 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123098.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123098.log.gz 2024-08-25T21:09:41.546 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38995.log 2024-08-25T21:09:41.547 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101101.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101101.log.gz 2024-08-25T21:09:41.547 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82222.log 2024-08-25T21:09:41.548 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38995.log.gz 2024-08-25T21:09:41.548 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52032.log 2024-08-25T21:09:41.549 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82222.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68900.log 2024-08-25T21:09:41.549 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82222.log.gz 2024-08-25T21:09:41.550 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52032.log.gz 2024-08-25T21:09:41.550 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118401.log 2024-08-25T21:09:41.550 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.68900.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.68900.log.gz 2024-08-25T21:09:41.550 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70806.log 2024-08-25T21:09:41.551 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.118401.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33741.log 2024-08-25T21:09:41.551 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118401.log.gz 2024-08-25T21:09:41.552 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70806.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70806.log.gz 2024-08-25T21:09:41.552 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110263.log 2024-08-25T21:09:41.553 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33741.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33741.log.gz 2024-08-25T21:09:41.553 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62054.log 2024-08-25T21:09:41.553 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110263.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110263.log.gz 2024-08-25T21:09:41.553 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27722.log 2024-08-25T21:09:41.554 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62054.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62054.log.gz 2024-08-25T21:09:41.554 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85834.log 2024-08-25T21:09:41.555 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27722.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27722.log.gz 2024-08-25T21:09:41.555 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40006.log 2024-08-25T21:09:41.556 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85834.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101805.log 2024-08-25T21:09:41.556 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85834.log.gz 2024-08-25T21:09:41.557 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40006.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40006.log.gz 2024-08-25T21:09:41.557 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32027.log 2024-08-25T21:09:41.557 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101805.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109905.log 2024-08-25T21:09:41.558 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101805.log.gz 2024-08-25T21:09:41.558 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32027.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32027.log.gz 2024-08-25T21:09:41.558 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113968.log 2024-08-25T21:09:41.559 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109905.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.109905.log.gz 2024-08-25T21:09:41.559 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109003.log 2024-08-25T21:09:41.560 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113968.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113968.log.gz 2024-08-25T21:09:41.560 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67670.log 2024-08-25T21:09:41.561 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109003.log.gz 2024-08-25T21:09:41.561 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47586.log 2024-08-25T21:09:41.562 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67670.log.gz 2024-08-25T21:09:41.562 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132148.log 2024-08-25T21:09:41.562 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47586.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47586.log.gz 2024-08-25T21:09:41.563 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35481.log 2024-08-25T21:09:41.563 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132148.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132148.log.gz 2024-08-25T21:09:41.564 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115586.log 2024-08-25T21:09:41.564 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.35481.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35481.log.gz 2024-08-25T21:09:41.565 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90702.log 2024-08-25T21:09:41.565 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115586.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.115586.log.gz --verbose 2024-08-25T21:09:41.565 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster2-client.mirror.58130.log 2024-08-25T21:09:41.566 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90702.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90702.log.gz 2024-08-25T21:09:41.566 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108836.log 2024-08-25T21:09:41.567 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58130.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58130.log.gz 2024-08-25T21:09:41.567 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75219.log 2024-08-25T21:09:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108836.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108836.log.gz 2024-08-25T21:09:41.568 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79261.log 2024-08-25T21:09:41.569 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75219.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66802.log 2024-08-25T21:09:41.569 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75219.log.gz 2024-08-25T21:09:41.569 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79261.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79261.log.gz 2024-08-25T21:09:41.569 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129429.log 2024-08-25T21:09:41.570 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66802.log.gz 2024-08-25T21:09:41.570 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51362.log 2024-08-25T21:09:41.571 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129429.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129429.log.gz 2024-08-25T21:09:41.571 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104910.log 2024-08-25T21:09:41.572 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51362.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51362.log.gz 2024-08-25T21:09:41.572 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.142247.log 2024-08-25T21:09:41.573 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104910.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104910.log.gz 2024-08-25T21:09:41.573 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130040.log 2024-08-25T21:09:41.573 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.142247.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.142247.log.gz 2024-08-25T21:09:41.574 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98657.log 2024-08-25T21:09:41.574 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.130040.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.95666.log 2024-08-25T21:09:41.575 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130040.log.gz 2024-08-25T21:09:41.575 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98657.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81011.log 2024-08-25T21:09:41.575 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98657.log.gz 2024-08-25T21:09:41.576 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.95666.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95666.log.gz 2024-08-25T21:09:41.576 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94901.log 2024-08-25T21:09:41.577 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81011.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81011.log.gz 2024-08-25T21:09:41.577 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30622.log 2024-08-25T21:09:41.577 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94901.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.73978.log -- replaced with /var/log/ceph/cluster1-client.mirror.94901.log.gz 2024-08-25T21:09:41.578 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:41.578 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30622.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30622.log.gz 2024-08-25T21:09:41.578 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69894.log 2024-08-25T21:09:41.579 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73978.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73978.log.gz 2024-08-25T21:09:41.579 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26759.log 2024-08-25T21:09:41.580 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.69894.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.69894.log.gz 2024-08-25T21:09:41.580 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.57552.log 2024-08-25T21:09:41.580 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26759.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26759.log.gz 2024-08-25T21:09:41.581 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.129792.log 2024-08-25T21:09:41.582 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57552.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57552.log.gz 2024-08-25T21:09:41.582 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44804.log 2024-08-25T21:09:41.582 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.129792.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.129792.log.gz 2024-08-25T21:09:41.582 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.23078.log 2024-08-25T21:09:41.583 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.44804.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85288.log 2024-08-25T21:09:41.583 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44804.log.gz 2024-08-25T21:09:41.584 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.23078.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.23078.log.gz 2024-08-25T21:09:41.584 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136748.log 2024-08-25T21:09:41.585 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85288.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85288.log.gz 2024-08-25T21:09:41.585 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82632.log 2024-08-25T21:09:41.586 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136748.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136748.log.gz 2024-08-25T21:09:41.586 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52077.log 2024-08-25T21:09:41.587 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82632.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73094.log 2024-08-25T21:09:41.587 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82632.log.gz 2024-08-25T21:09:41.587 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52077.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52077.log.gz 2024-08-25T21:09:41.587 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109779.log 2024-08-25T21:09:41.588 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73094.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73094.log.gz 2024-08-25T21:09:41.588 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42861.log 2024-08-25T21:09:41.589 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109779.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109779.log.gz 2024-08-25T21:09:41.589 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37652.log 2024-08-25T21:09:41.590 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42861.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114389.log 2024-08-25T21:09:41.590 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42861.log.gz 2024-08-25T21:09:41.590 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37652.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.37652.log.gz -- 2024-08-25T21:09:41.590 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.121883.log 2024-08-25T21:09:41.591 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114389.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.114389.log.gz -- 2024-08-25T21:09:41.591 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.70958.log 2024-08-25T21:09:41.592 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121883.log.gz 2024-08-25T21:09:41.592 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34229.log 2024-08-25T21:09:41.593 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70958.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34397.log 2024-08-25T21:09:41.593 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70958.log.gz 2024-08-25T21:09:41.594 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34229.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34229.log.gz 2024-08-25T21:09:41.594 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44569.log 2024-08-25T21:09:41.594 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34397.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.34397.log.gz -5 2024-08-25T21:09:41.595 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.73750.log 2024-08-25T21:09:41.595 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44569.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44569.log.gz 2024-08-25T21:09:41.595 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105498.log 2024-08-25T21:09:41.596 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73750.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37525.log 2024-08-25T21:09:41.596 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73750.log.gz 2024-08-25T21:09:41.597 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105498.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105498.log.gz 2024-08-25T21:09:41.597 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136316.log 2024-08-25T21:09:41.598 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37525.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37525.log.gz 2024-08-25T21:09:41.598 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57456.log 2024-08-25T21:09:41.599 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136316.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.95055.log 2024-08-25T21:09:41.599 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136316.log.gz 2024-08-25T21:09:41.599 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.57456.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57456.log.gz 2024-08-25T21:09:41.599 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81746.log 2024-08-25T21:09:41.600 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.95055.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80029.log 2024-08-25T21:09:41.600 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95055.log.gz 2024-08-25T21:09:41.601 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.81746.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81746.log.gz 2024-08-25T21:09:41.601 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113966.log 2024-08-25T21:09:41.602 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80029.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80029.log.gz 2024-08-25T21:09:41.602 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132310.log 2024-08-25T21:09:41.603 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113966.log.gz 2024-08-25T21:09:41.603 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115585.log 2024-08-25T21:09:41.603 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132310.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132310.log.gz 2024-08-25T21:09:41.604 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108807.log 2024-08-25T21:09:41.604 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115585.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115585.log.gz 2024-08-25T21:09:41.604 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134759.log 2024-08-25T21:09:41.605 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108807.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108807.log.gz 2024-08-25T21:09:41.605 INFO:teuthology.orchestra.run.smithi121.stderr: 92.6% -- replaced with /var/log/ceph/cluster2.audit.log.gz 2024-08-25T21:09:41.605 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65916.log 2024-08-25T21:09:41.606 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134759.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134759.log.gz 2024-08-25T21:09:41.606 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78551.log 2024-08-25T21:09:41.607 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65916.log.gz 2024-08-25T21:09:41.607 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25882.log 2024-08-25T21:09:41.608 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78551.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78551.log.gz 2024-08-25T21:09:41.608 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111891.log 2024-08-25T21:09:41.608 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25882.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25882.log.gz 2024-08-25T21:09:41.609 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124606.log 2024-08-25T21:09:41.609 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111891.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111891.log.gz 2024-08-25T21:09:41.609 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39711.log 2024-08-25T21:09:41.610 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124606.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124606.log.gz 2024-08-25T21:09:41.610 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105805.log 2024-08-25T21:09:41.611 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39711.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39711.log.gz 2024-08-25T21:09:41.611 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65582.log 2024-08-25T21:09:41.612 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.105805.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105805.log.gz 2024-08-25T21:09:41.612 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53858.log 2024-08-25T21:09:41.612 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65582.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122490.log 2024-08-25T21:09:41.613 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65582.log.gz 2024-08-25T21:09:41.613 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53858.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53858.log.gz 2024-08-25T21:09:41.613 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122721.log 2024-08-25T21:09:41.614 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122490.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122490.log.gz 2024-08-25T21:09:41.614 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89406.log 2024-08-25T21:09:41.615 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122721.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122721.log.gz 2024-08-25T21:09:41.615 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93642.log 2024-08-25T21:09:41.616 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89406.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89406.log.gz 2024-08-25T21:09:41.616 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53463.log 2024-08-25T21:09:41.616 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.93642.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93642.log.gz 2024-08-25T21:09:41.616 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141631.log 2024-08-25T21:09:41.617 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53463.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53463.log.gz 2024-08-25T21:09:41.617 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23686.log 2024-08-25T21:09:41.618 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141631.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141631.log.gz 2024-08-25T21:09:41.618 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138277.log 2024-08-25T21:09:41.619 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23686.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23686.log.gz 2024-08-25T21:09:41.619 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75554.log 2024-08-25T21:09:41.620 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138277.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138277.log.gz 2024-08-25T21:09:41.620 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53127.log 2024-08-25T21:09:41.621 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75554.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75554.log.gz 2024-08-25T21:09:41.621 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109360.log 2024-08-25T21:09:41.621 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53127.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53127.log.gz 2024-08-25T21:09:41.621 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108566.log 2024-08-25T21:09:41.622 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109360.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109360.log.gz 2024-08-25T21:09:41.622 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66906.log 2024-08-25T21:09:41.623 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108566.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110055.log 2024-08-25T21:09:41.623 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108566.log.gz 2024-08-25T21:09:41.624 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66906.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66906.log.gz 2024-08-25T21:09:41.624 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95775.log 2024-08-25T21:09:41.624 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110055.log.gz 2024-08-25T21:09:41.625 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35458.log 2024-08-25T21:09:41.625 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95775.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.95775.log.gz 2024-08-25T21:09:41.625 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82942.log 2024-08-25T21:09:41.626 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35458.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35458.log.gz 2024-08-25T21:09:41.626 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139796.log 2024-08-25T21:09:41.627 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82942.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82942.log.gz -5 2024-08-25T21:09:41.627 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.65538.log 2024-08-25T21:09:41.628 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139796.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139796.log.gz 2024-08-25T21:09:41.628 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42735.log 2024-08-25T21:09:41.628 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65538.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65538.log.gz 2024-08-25T21:09:41.628 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55950.log 2024-08-25T21:09:41.629 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42735.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42735.log.gz 2024-08-25T21:09:41.629 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137170.log 2024-08-25T21:09:41.630 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55950.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55950.log.gz 2024-08-25T21:09:41.630 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110922.log 2024-08-25T21:09:41.631 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137170.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137170.log.gz 2024-08-25T21:09:41.631 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54705.log 2024-08-25T21:09:41.631 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110922.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110922.log.gz 2024-08-25T21:09:41.631 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66046.log 2024-08-25T21:09:41.632 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54705.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54705.log.gz 2024-08-25T21:09:41.632 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69821.log 2024-08-25T21:09:41.633 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66046.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66046.log.gz 2024-08-25T21:09:41.633 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133110.log 2024-08-25T21:09:41.634 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.69821.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.admin.69821.log.gz 2024-08-25T21:09:41.634 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.admin.54564.log 2024-08-25T21:09:41.634 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133110.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133110.log.gz 2024-08-25T21:09:41.635 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123251.log 2024-08-25T21:09:41.635 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54564.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54564.log.gz 2024-08-25T21:09:41.635 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75707.log 2024-08-25T21:09:41.636 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123251.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123251.log.gz 2024-08-25T21:09:41.636 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120643.log 2024-08-25T21:09:41.637 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75707.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75707.log.gz 2024-08-25T21:09:41.637 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70371.log 2024-08-25T21:09:41.638 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120643.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120643.log.gz 2024-08-25T21:09:41.638 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35158.log 2024-08-25T21:09:41.638 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70371.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70371.log.gz 2024-08-25T21:09:41.639 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94501.log 2024-08-25T21:09:41.639 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35158.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35158.log.gz 2024-08-25T21:09:41.639 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84616.log 2024-08-25T21:09:41.640 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94501.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94501.log.gz 2024-08-25T21:09:41.640 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62817.log 2024-08-25T21:09:41.641 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84616.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84616.log.gz 2024-08-25T21:09:41.641 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112693.log 2024-08-25T21:09:41.642 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62817.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62817.log.gz 2024-08-25T21:09:41.642 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106775.log 2024-08-25T21:09:41.642 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112693.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112693.log.gz 2024-08-25T21:09:41.643 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120603.log 2024-08-25T21:09:41.643 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106775.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106775.log.gz 2024-08-25T21:09:41.643 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64616.log 2024-08-25T21:09:41.644 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120603.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120603.log.gz 2024-08-25T21:09:41.644 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37611.log 2024-08-25T21:09:41.645 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64616.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64616.log.gz 2024-08-25T21:09:41.645 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82243.log 2024-08-25T21:09:41.645 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37611.log.gz 2024-08-25T21:09:41.645 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59209.log 2024-08-25T21:09:41.647 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107786.log 2024-08-25T21:09:41.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82243.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82243.log.gz 2024-08-25T21:09:41.647 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59209.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59209.log.gz 2024-08-25T21:09:41.647 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51654.log 2024-08-25T21:09:41.648 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.107786.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107786.log.gz 2024-08-25T21:09:41.648 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70257.log 2024-08-25T21:09:41.649 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51654.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51654.log.gz 2024-08-25T21:09:41.649 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59077.log 2024-08-25T21:09:41.650 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70257.log.gz 2024-08-25T21:09:41.650 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63019.log 2024-08-25T21:09:41.650 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59077.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75552.log 2024-08-25T21:09:41.651 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59077.log.gz 2024-08-25T21:09:41.652 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63019.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56177.log 2024-08-25T21:09:41.652 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63019.log.gz 2024-08-25T21:09:41.652 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75552.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75552.log.gz 2024-08-25T21:09:41.652 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73143.log 2024-08-25T21:09:41.653 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56177.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56177.log.gz 2024-08-25T21:09:41.653 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100588.log 2024-08-25T21:09:41.654 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73143.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73143.log.gz 2024-08-25T21:09:41.654 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108877.log 2024-08-25T21:09:41.654 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100588.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100588.log.gz 2024-08-25T21:09:41.655 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131041.log 2024-08-25T21:09:41.655 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108877.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108877.log.gz 2024-08-25T21:09:41.655 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103488.log 2024-08-25T21:09:41.656 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131041.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131041.log.gz 2024-08-25T21:09:41.656 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50186.log 2024-08-25T21:09:41.657 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103488.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103488.log.gz 2024-08-25T21:09:41.657 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56884.log 2024-08-25T21:09:41.658 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50186.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50186.log.gz 2024-08-25T21:09:41.658 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134036.log 2024-08-25T21:09:41.658 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.56884.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56884.log.gz 2024-08-25T21:09:41.659 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107222.log 2024-08-25T21:09:41.659 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134036.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134036.log.gz 2024-08-25T21:09:41.660 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33531.log 2024-08-25T21:09:41.660 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107222.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107222.log.gz 2024-08-25T21:09:41.660 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36690.log 2024-08-25T21:09:41.661 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33531.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33531.log.gz 2024-08-25T21:09:41.661 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91270.log 2024-08-25T21:09:41.662 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36690.log.gz 2024-08-25T21:09:41.662 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140443.log 2024-08-25T21:09:41.663 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91270.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91270.log.gz 2024-08-25T21:09:41.663 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133577.log 2024-08-25T21:09:41.664 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.140443.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140443.log.gzgzip 2024-08-25T21:09:41.664 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117925.log 2024-08-25T21:09:41.665 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133577.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133577.log.gz 2024-08-25T21:09:41.665 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27303.log 2024-08-25T21:09:41.665 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.117925.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117925.log.gz 2024-08-25T21:09:41.666 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.22839.log 2024-08-25T21:09:41.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27303.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27303.log.gz 2024-08-25T21:09:41.666 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121249.log 2024-08-25T21:09:41.667 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.22839.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111133.log 2024-08-25T21:09:41.667 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.22839.log.gz 2024-08-25T21:09:41.668 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121249.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121249.log.gz 2024-08-25T21:09:41.668 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34188.log 2024-08-25T21:09:41.669 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111133.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111133.log.gz 2024-08-25T21:09:41.669 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62472.log 2024-08-25T21:09:41.669 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34188.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100940.log 2024-08-25T21:09:41.669 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34188.log.gz 2024-08-25T21:09:41.670 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62472.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62472.log.gz 2024-08-25T21:09:41.670 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92723.log 2024-08-25T21:09:41.671 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100940.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100940.log.gz 2024-08-25T21:09:41.671 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60608.log 2024-08-25T21:09:41.672 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92723.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92723.log.gz 2024-08-25T21:09:41.672 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84123.log 2024-08-25T21:09:41.673 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60608.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60608.log.gz 2024-08-25T21:09:41.673 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49178.log 2024-08-25T21:09:41.674 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84123.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84123.log.gz 2024-08-25T21:09:41.674 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93114.log 2024-08-25T21:09:41.675 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49178.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49178.log.gz 2024-08-25T21:09:41.675 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110964.log 2024-08-25T21:09:41.675 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.93114.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93114.log.gz 2024-08-25T21:09:41.675 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125536.log 2024-08-25T21:09:41.676 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110964.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108961.log 2024-08-25T21:09:41.676 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110964.log.gz 2024-08-25T21:09:41.677 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125536.log.gz 2024-08-25T21:09:41.677 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.22655.log 2024-08-25T21:09:41.678 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108961.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108961.log.gz -5 2024-08-25T21:09:41.678 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.90522.log 2024-08-25T21:09:41.679 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.22655.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.22655.log.gz 2024-08-25T21:09:41.679 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121420.log 2024-08-25T21:09:41.680 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90522.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90522.log.gz 2024-08-25T21:09:41.680 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59816.log 2024-08-25T21:09:41.680 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121420.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.24348.log 2024-08-25T21:09:41.680 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121420.log.gz 2024-08-25T21:09:41.681 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59816.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59816.log.gz 2024-08-25T21:09:41.681 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40803.log 2024-08-25T21:09:41.682 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.24348.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.24348.log.gz 2024-08-25T21:09:41.682 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88205.log 2024-08-25T21:09:41.683 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40803.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111173.log 2024-08-25T21:09:41.683 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40803.log.gz 2024-08-25T21:09:41.683 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88205.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88205.log.gz 2024-08-25T21:09:41.684 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81601.log 2024-08-25T21:09:41.684 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111173.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111173.log.gz 2024-08-25T21:09:41.684 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.130207.log 2024-08-25T21:09:41.685 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81601.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81601.log.gz 2024-08-25T21:09:41.685 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133043.log 2024-08-25T21:09:41.686 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.130207.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.130207.log.gz 2024-08-25T21:09:41.686 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70713.log 2024-08-25T21:09:41.687 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133043.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133043.log.gz 2024-08-25T21:09:41.687 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80722.log 2024-08-25T21:09:41.688 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70713.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70713.log.gz 2024-08-25T21:09:41.688 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85246.log 2024-08-25T21:09:41.689 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80722.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62389.log 2024-08-25T21:09:41.689 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80722.log.gz 2024-08-25T21:09:41.689 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85246.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85246.log.gz 2024-08-25T21:09:41.689 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28687.log 2024-08-25T21:09:41.690 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62389.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62389.log.gz 2024-08-25T21:09:41.690 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51001.log 2024-08-25T21:09:41.691 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28687.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.130554.log -- replaced with /var/log/ceph/cluster1-client.mirror.28687.log.gz 2024-08-25T21:09:41.691 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:41.691 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51001.log.gz 2024-08-25T21:09:41.692 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76520.log 2024-08-25T21:09:41.692 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130554.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130554.log.gz 2024-08-25T21:09:41.693 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84827.log 2024-08-25T21:09:41.693 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76520.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76520.log.gz 2024-08-25T21:09:41.693 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31331.log 2024-08-25T21:09:41.694 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84827.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84827.log.gz 2024-08-25T21:09:41.694 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57902.log 2024-08-25T21:09:41.695 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31331.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31331.log.gz 2024-08-25T21:09:41.695 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49806.log 2024-08-25T21:09:41.696 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.57902.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57902.log.gz 2024-08-25T21:09:41.696 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111492.log 2024-08-25T21:09:41.697 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49806.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49806.log.gz 2024-08-25T21:09:41.697 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30307.log 2024-08-25T21:09:41.698 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111492.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93844.log 2024-08-25T21:09:41.698 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111492.log.gz 2024-08-25T21:09:41.698 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30307.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30307.log.gz 2024-08-25T21:09:41.699 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95139.log 2024-08-25T21:09:41.699 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.93844.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93844.log.gz 2024-08-25T21:09:41.699 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84575.log 2024-08-25T21:09:41.700 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95139.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95139.log.gz 2024-08-25T21:09:41.700 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134780.log 2024-08-25T21:09:41.701 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84575.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84575.log.gz 2024-08-25T21:09:41.701 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52705.log 2024-08-25T21:09:41.702 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134780.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134780.log.gz 2024-08-25T21:09:41.702 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59078.log 2024-08-25T21:09:41.702 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52705.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52705.log.gz 2024-08-25T21:09:41.703 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90970.log 2024-08-25T21:09:41.703 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59078.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59078.log.gz 2024-08-25T21:09:41.703 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31498.log 2024-08-25T21:09:41.704 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90970.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90970.log.gz 2024-08-25T21:09:41.704 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105764.log 2024-08-25T21:09:41.705 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31498.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31498.log.gz 2024-08-25T21:09:41.705 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137762.log 2024-08-25T21:09:41.706 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105764.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105764.log.gz 2024-08-25T21:09:41.706 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136899.log 2024-08-25T21:09:41.706 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137762.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137762.log.gz 2024-08-25T21:09:41.707 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77903.log 2024-08-25T21:09:41.707 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136899.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136899.log.gz 2024-08-25T21:09:41.707 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125326.log 2024-08-25T21:09:41.708 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77903.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77903.log.gz 2024-08-25T21:09:41.708 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71542.log 2024-08-25T21:09:41.709 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125326.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125326.log.gz 2024-08-25T21:09:41.709 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75831.log 2024-08-25T21:09:41.710 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71542.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71542.log.gz 2024-08-25T21:09:41.710 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29833.log 2024-08-25T21:09:41.710 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75831.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75831.log.gz 2024-08-25T21:09:41.711 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140793.log 2024-08-25T21:09:41.712 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29833.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29833.log.gz 2024-08-25T21:09:41.712 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115474.log 2024-08-25T21:09:41.713 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140793.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140793.log.gz 2024-08-25T21:09:41.713 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50080.log 2024-08-25T21:09:41.714 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115474.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115474.log.gz 2024-08-25T21:09:41.714 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51528.log 2024-08-25T21:09:41.714 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50080.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141791.log 2024-08-25T21:09:41.714 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50080.log.gz 2024-08-25T21:09:41.715 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51528.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51528.log.gz 2024-08-25T21:09:41.715 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117478.log 2024-08-25T21:09:41.716 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141791.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119828.log 2024-08-25T21:09:41.716 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141791.log.gz 2024-08-25T21:09:41.717 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117478.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117478.log.gz 2024-08-25T21:09:41.717 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60151.log 2024-08-25T21:09:41.718 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119828.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119828.log.gz 2024-08-25T21:09:41.718 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40215.log 2024-08-25T21:09:41.718 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60151.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60151.log.gz 2024-08-25T21:09:41.718 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101092.log 2024-08-25T21:09:41.719 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40215.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40215.log.gz 2024-08-25T21:09:41.719 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48198.log 2024-08-25T21:09:41.720 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101092.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101092.log.gz 2024-08-25T21:09:41.720 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136125.log 2024-08-25T21:09:41.721 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48198.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48198.log.gz 2024-08-25T21:09:41.721 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131250.log 2024-08-25T21:09:41.722 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136125.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136125.log.gz 2024-08-25T21:09:41.722 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93870.log 2024-08-25T21:09:41.723 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131250.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131250.log.gz 2024-08-25T21:09:41.723 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78374.log 2024-08-25T21:09:41.724 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.93870.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93870.log.gz 2024-08-25T21:09:41.724 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48133.log 2024-08-25T21:09:41.725 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78374.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78374.log.gz 2024-08-25T21:09:41.725 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66801.log 2024-08-25T21:09:41.725 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48133.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48133.log.gz 2024-08-25T21:09:41.725 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82088.log 2024-08-25T21:09:41.726 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66801.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66801.log.gz 2024-08-25T21:09:41.726 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73958.log 2024-08-25T21:09:41.727 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82088.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster2-client.mirror.82088.log.gz --verbose 2024-08-25T21:09:41.727 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.99095.log 2024-08-25T21:09:41.728 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73958.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73958.log.gz 2024-08-25T21:09:41.728 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93419.log 2024-08-25T21:09:41.728 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99095.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133192.log 2024-08-25T21:09:41.728 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99095.log.gz 2024-08-25T21:09:41.729 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93419.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93419.log.gz 2024-08-25T21:09:41.729 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72136.log 2024-08-25T21:09:41.730 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133192.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42525.log 2024-08-25T21:09:41.730 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133192.log.gz 2024-08-25T21:09:41.731 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72136.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72136.log.gz 2024-08-25T21:09:41.731 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110348.log 2024-08-25T21:09:41.732 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42525.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42525.log.gz 2024-08-25T21:09:41.732 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73284.log 2024-08-25T21:09:41.732 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110348.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110348.log.gz 2024-08-25T21:09:41.732 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116970.log 2024-08-25T21:09:41.733 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73284.log.gz 2024-08-25T21:09:41.733 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26882.log 2024-08-25T21:09:41.734 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.116970.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.116970.log.gz 2024-08-25T21:09:41.734 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58908.log 2024-08-25T21:09:41.735 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26882.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26882.log.gz 2024-08-25T21:09:41.735 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65160.log 2024-08-25T21:09:41.735 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58908.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58908.log.gz 2024-08-25T21:09:41.736 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58855.log 2024-08-25T21:09:41.736 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65160.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65160.log.gz 2024-08-25T21:09:41.737 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94528.log 2024-08-25T21:09:41.737 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58855.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58855.log.gz 2024-08-25T21:09:41.738 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125118.log 2024-08-25T21:09:41.738 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94528.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94528.log.gz 2024-08-25T21:09:41.738 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126061.log 2024-08-25T21:09:41.739 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125118.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125118.log.gz 2024-08-25T21:09:41.740 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51326.log 2024-08-25T21:09:41.740 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.126061.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126061.log.gz 2024-08-25T21:09:41.740 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102548.log 2024-08-25T21:09:41.741 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51326.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51326.log.gz 2024-08-25T21:09:41.741 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136544.log 2024-08-25T21:09:41.742 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102548.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102548.log.gz 2024-08-25T21:09:41.742 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94642.log 2024-08-25T21:09:41.743 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136544.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64446.log 2024-08-25T21:09:41.743 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136544.log.gz 2024-08-25T21:09:41.744 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94642.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94642.log.gz 2024-08-25T21:09:41.744 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25589.log 2024-08-25T21:09:41.745 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64446.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64446.log.gz 2024-08-25T21:09:41.745 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49302.log 2024-08-25T21:09:41.745 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25589.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.71657.log 2024-08-25T21:09:41.745 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.25589.log.gz 2024-08-25T21:09:41.746 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49302.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49302.log.gz 2024-08-25T21:09:41.746 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141584.log 2024-08-25T21:09:41.747 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71657.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71657.log.gz 2024-08-25T21:09:41.747 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76188.log 2024-08-25T21:09:41.747 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.141584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141584.log.gz 2024-08-25T21:09:41.748 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107091.log 2024-08-25T21:09:41.748 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76188.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42274.log 2024-08-25T21:09:41.748 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76188.log.gz 2024-08-25T21:09:41.749 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107091.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107091.log.gz 2024-08-25T21:09:41.749 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104362.log 2024-08-25T21:09:41.750 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42274.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42274.log.gz 2024-08-25T21:09:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81346.log 2024-08-25T21:09:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104362.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104362.log.gz 2024-08-25T21:09:41.751 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66685.log 2024-08-25T21:09:41.752 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.81346.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81346.log.gz 2024-08-25T21:09:41.752 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70786.log 2024-08-25T21:09:41.752 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66685.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66685.log.gz 2024-08-25T21:09:41.753 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54365.log 2024-08-25T21:09:41.753 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70786.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70786.log.gz 2024-08-25T21:09:41.753 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61129.log 2024-08-25T21:09:41.754 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54365.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54365.log.gz 2024-08-25T21:09:41.755 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69734.log 2024-08-25T21:09:41.755 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61129.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61129.log.gz 2024-08-25T21:09:41.755 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29130.log 2024-08-25T21:09:41.756 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.69734.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69734.log.gz 2024-08-25T21:09:41.756 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68921.log 2024-08-25T21:09:41.757 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29130.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135223.log 2024-08-25T21:09:41.757 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29130.log.gz 2024-08-25T21:09:41.757 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.68921.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster2-client.mirror.68921.log.gz --verbose 2024-08-25T21:09:41.758 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.admin.59865.log 2024-08-25T21:09:41.758 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135223.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135223.log.gz 2024-08-25T21:09:41.758 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137471.log 2024-08-25T21:09:41.759 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.59865.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59865.log.gz 2024-08-25T21:09:41.759 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83005.log 2024-08-25T21:09:41.760 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137471.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137471.log.gz 2024-08-25T21:09:41.760 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55849.log 2024-08-25T21:09:41.761 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83005.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83005.log.gz 2024-08-25T21:09:41.761 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48369.log 2024-08-25T21:09:41.762 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55849.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55849.log.gz 2024-08-25T21:09:41.762 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70350.log 2024-08-25T21:09:41.762 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48369.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.136416.log 2024-08-25T21:09:41.763 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.48369.log.gz 2024-08-25T21:09:41.763 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70350.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70350.log.gz 2024-08-25T21:09:41.763 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38786.log 2024-08-25T21:09:41.764 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136416.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117557.log 2024-08-25T21:09:41.764 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136416.log.gz 2024-08-25T21:09:41.765 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38786.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38786.log.gz 2024-08-25T21:09:41.765 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87866.log 2024-08-25T21:09:41.766 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.117557.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.129906.log 2024-08-25T21:09:41.766 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117557.log.gz 2024-08-25T21:09:41.766 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87866.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66882.log 2024-08-25T21:09:41.766 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87866.log.gz 2024-08-25T21:09:41.767 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.129906.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.129906.log.gz 2024-08-25T21:09:41.767 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112773.log 2024-08-25T21:09:41.768 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66882.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66882.log.gz 2024-08-25T21:09:41.768 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22925.log 2024-08-25T21:09:41.769 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112773.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112773.log.gz 2024-08-25T21:09:41.769 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90424.log 2024-08-25T21:09:41.769 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22925.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22925.log.gz 2024-08-25T21:09:41.770 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39541.log 2024-08-25T21:09:41.770 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90424.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90424.log.gz 2024-08-25T21:09:41.771 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108461.log 2024-08-25T21:09:41.772 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39541.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39541.log.gz 2024-08-25T21:09:41.772 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138108.log 2024-08-25T21:09:41.772 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108461.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108461.log.gz 2024-08-25T21:09:41.773 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47979.log 2024-08-25T21:09:41.773 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138108.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.136756.log 2024-08-25T21:09:41.773 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.138108.log.gz 2024-08-25T21:09:41.774 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47979.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47979.log.gz 2024-08-25T21:09:41.774 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85686.log 2024-08-25T21:09:41.775 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136756.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103648.log 2024-08-25T21:09:41.775 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136756.log.gz 2024-08-25T21:09:41.776 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85686.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66532.log 2024-08-25T21:09:41.776 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85686.log.gz 2024-08-25T21:09:41.776 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103648.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103648.log.gz 2024-08-25T21:09:41.777 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75368.log 2024-08-25T21:09:41.777 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66532.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66532.log.gz 2024-08-25T21:09:41.777 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87766.log 2024-08-25T21:09:41.778 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75368.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75368.log.gz 2024-08-25T21:09:41.778 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117054.log 2024-08-25T21:09:41.779 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87766.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87766.log.gz 2024-08-25T21:09:41.779 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30601.log 2024-08-25T21:09:41.780 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117054.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83073.log 2024-08-25T21:09:41.780 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117054.log.gz 2024-08-25T21:09:41.780 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30601.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69630.log 2024-08-25T21:09:41.781 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30601.log.gz 2024-08-25T21:09:41.781 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83073.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83073.log.gz 2024-08-25T21:09:41.781 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78245.log 2024-08-25T21:09:41.782 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69630.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69630.log.gz 2024-08-25T21:09:41.782 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123854.log 2024-08-25T21:09:41.783 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78245.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78245.log.gz 2024-08-25T21:09:41.783 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100315.log 2024-08-25T21:09:41.784 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123854.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123854.log.gz 2024-08-25T21:09:41.784 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88870.log 2024-08-25T21:09:41.785 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100315.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100315.log.gz 2024-08-25T21:09:41.785 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.21930.log 2024-08-25T21:09:41.785 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88870.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88870.log.gz 2024-08-25T21:09:41.786 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36618.log 2024-08-25T21:09:41.786 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.21930.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.21930.log.gz 2024-08-25T21:09:41.786 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123308.log 2024-08-25T21:09:41.787 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36618.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36618.log.gz 2024-08-25T21:09:41.787 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27031.log 2024-08-25T21:09:41.788 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123308.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123308.log.gz 2024-08-25T21:09:41.788 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45991.log 2024-08-25T21:09:41.789 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27031.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27031.log.gz 2024-08-25T21:09:41.789 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58741.log 2024-08-25T21:09:41.790 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.45991.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45991.log.gz 2024-08-25T21:09:41.790 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67604.log 2024-08-25T21:09:41.791 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58741.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58741.log.gz 2024-08-25T21:09:41.791 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133679.log 2024-08-25T21:09:41.792 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67604.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67604.log.gz 2024-08-25T21:09:41.792 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132646.log 2024-08-25T21:09:41.793 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133679.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133679.log.gz 2024-08-25T21:09:41.793 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54699.log 2024-08-25T21:09:41.793 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132646.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132646.log.gz 2024-08-25T21:09:41.794 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134733.log 2024-08-25T21:09:41.794 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54699.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54699.log.gz 2024-08-25T21:09:41.794 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61398.log 2024-08-25T21:09:41.795 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134733.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134733.log.gz 2024-08-25T21:09:41.795 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61441.log 2024-08-25T21:09:41.796 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61398.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61398.log.gz 2024-08-25T21:09:41.796 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26339.log 2024-08-25T21:09:41.797 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61441.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61441.log.gz 2024-08-25T21:09:41.797 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50984.log 2024-08-25T21:09:41.797 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26339.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.26339.log.gz 2024-08-25T21:09:41.797 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.admin.79330.log 2024-08-25T21:09:41.798 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50984.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50984.log.gz 2024-08-25T21:09:41.798 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122595.log 2024-08-25T21:09:41.799 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79330.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79330.log.gz 2024-08-25T21:09:41.799 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77675.log 2024-08-25T21:09:41.800 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122595.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122595.log.gz 2024-08-25T21:09:41.800 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109738.log 2024-08-25T21:09:41.801 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77675.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77675.log.gz 2024-08-25T21:09:41.801 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137024.log 2024-08-25T21:09:41.801 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61570.log 2024-08-25T21:09:41.802 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109738.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109738.log.gz 2024-08-25T21:09:41.802 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137024.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137024.log.gz 2024-08-25T21:09:41.802 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90929.log 2024-08-25T21:09:41.803 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61570.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61570.log.gz 2024-08-25T21:09:41.803 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94838.log 2024-08-25T21:09:41.804 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90929.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90929.log.gz 2024-08-25T21:09:41.804 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70008.log 2024-08-25T21:09:41.805 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94838.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94838.log.gz 2024-08-25T21:09:41.805 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140942.log 2024-08-25T21:09:41.805 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70008.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70008.log.gz 2024-08-25T21:09:41.805 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103780.log 2024-08-25T21:09:41.806 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.140942.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140942.log.gz 2024-08-25T21:09:41.806 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131102.log 2024-08-25T21:09:41.807 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103780.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103780.log.gz 2024-08-25T21:09:41.807 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72965.log 2024-08-25T21:09:41.808 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131102.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131102.log.gz 2024-08-25T21:09:41.808 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103863.log 2024-08-25T21:09:41.809 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72965.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72965.log.gz 2024-08-25T21:09:41.809 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88752.log 2024-08-25T21:09:41.809 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103863.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103863.log.gz 2024-08-25T21:09:41.809 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125577.log 2024-08-25T21:09:41.810 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88752.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88752.log.gz 2024-08-25T21:09:41.810 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43961.log 2024-08-25T21:09:41.811 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125577.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125577.log.gz 2024-08-25T21:09:41.811 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36444.log 2024-08-25T21:09:41.812 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43961.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43961.log.gz 2024-08-25T21:09:41.812 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46132.log 2024-08-25T21:09:41.812 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36444.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36444.log.gz 2024-08-25T21:09:41.813 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76585.log 2024-08-25T21:09:41.813 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46132.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46132.log.gz 2024-08-25T21:09:41.813 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138199.log 2024-08-25T21:09:41.814 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76585.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76585.log.gz 2024-08-25T21:09:41.814 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46158.log 2024-08-25T21:09:41.815 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138199.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138199.log.gz 2024-08-25T21:09:41.815 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132020.log 2024-08-25T21:09:41.816 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46158.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46158.log.gz 2024-08-25T21:09:41.816 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142382.log 2024-08-25T21:09:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132020.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100356.log 2024-08-25T21:09:41.817 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132020.log.gz 2024-08-25T21:09:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.142382.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142382.log.gz 2024-08-25T21:09:41.818 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65414.log 2024-08-25T21:09:41.818 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100356.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100356.log.gz 2024-08-25T21:09:41.818 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.142425.log 2024-08-25T21:09:41.819 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65414.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65414.log.gz 2024-08-25T21:09:41.819 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129052.log 2024-08-25T21:09:41.825 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.142425.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.142425.log.gz 2024-08-25T21:09:41.825 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34001.log 2024-08-25T21:09:41.825 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129052.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129052.log.gz 2024-08-25T21:09:41.825 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54386.log 2024-08-25T21:09:41.825 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34001.log.gz 2024-08-25T21:09:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79416.log 2024-08-25T21:09:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54386.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54386.log.gz 2024-08-25T21:09:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105630.log 2024-08-25T21:09:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79416.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79416.log.gz 2024-08-25T21:09:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.17224.log 2024-08-25T21:09:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105630.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105630.log.gz 2024-08-25T21:09:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106355.log 2024-08-25T21:09:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.17224.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.17224.log.gz 2024-08-25T21:09:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66000.log 2024-08-25T21:09:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.106355.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106355.log.gz 2024-08-25T21:09:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28269.log 2024-08-25T21:09:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66000.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66000.log.gz 2024-08-25T21:09:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120708.log 2024-08-25T21:09:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28269.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52327.log 2024-08-25T21:09:41.828 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28269.log.gz 2024-08-25T21:09:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120708.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120708.log.gz 2024-08-25T21:09:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80177.log 2024-08-25T21:09:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52327.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52327.log.gz 2024-08-25T21:09:41.829 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106603.log 2024-08-25T21:09:41.829 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80177.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80177.log.gz 2024-08-25T21:09:41.829 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.59579.log 2024-08-25T21:09:41.830 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106603.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106603.log.gz 2024-08-25T21:09:41.830 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51160.log 2024-08-25T21:09:41.831 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.59579.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59579.log.gz 2024-08-25T21:09:41.831 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112667.log 2024-08-25T21:09:41.832 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51160.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124708.log 2024-08-25T21:09:41.832 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51160.log.gz 2024-08-25T21:09:41.832 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112667.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112667.log.gz 2024-08-25T21:09:41.832 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141563.log 2024-08-25T21:09:41.833 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124708.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124708.log.gz 2024-08-25T21:09:41.833 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52452.log 2024-08-25T21:09:41.834 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141563.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141563.log.gz 2024-08-25T21:09:41.834 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131623.log 2024-08-25T21:09:41.835 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52452.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52452.log.gz 2024-08-25T21:09:41.835 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71764.log 2024-08-25T21:09:41.836 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131623.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131623.log.gz 2024-08-25T21:09:41.836 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49933.log 2024-08-25T21:09:41.836 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.71764.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71764.log.gz 2024-08-25T21:09:41.837 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121523.log 2024-08-25T21:09:41.837 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49933.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70464.log 2024-08-25T21:09:41.837 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49933.log.gz 2024-08-25T21:09:41.838 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121523.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.121523.log.gz -5 2024-08-25T21:09:41.838 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.111134.log 2024-08-25T21:09:41.839 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70464.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49764.log 2024-08-25T21:09:41.839 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70464.log.gz 2024-08-25T21:09:41.839 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111134.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53420.log 2024-08-25T21:09:41.840 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111134.log.gz 2024-08-25T21:09:41.840 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49764.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34599.log 2024-08-25T21:09:41.840 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49764.log.gz 2024-08-25T21:09:41.841 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53420.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45787.log 2024-08-25T21:09:41.841 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53420.log.gz 2024-08-25T21:09:41.842 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34599.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34599.log.gz 2024-08-25T21:09:41.842 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98344.log 2024-08-25T21:09:41.843 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45787.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45787.log.gz 2024-08-25T21:09:41.843 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95298.log 2024-08-25T21:09:41.843 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98344.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98344.log.gz 2024-08-25T21:09:41.844 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118153.log 2024-08-25T21:09:41.845 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95298.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95298.log.gz 2024-08-25T21:09:41.845 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121815.log 2024-08-25T21:09:41.845 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.118153.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118153.log.gz 2024-08-25T21:09:41.846 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88418.log 2024-08-25T21:09:41.846 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121815.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121815.log.gz 2024-08-25T21:09:41.846 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112833.log 2024-08-25T21:09:41.847 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88418.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33195.log 2024-08-25T21:09:41.847 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88418.log.gz 2024-08-25T21:09:41.848 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112833.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112833.log.gz 2024-08-25T21:09:41.848 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100987.log 2024-08-25T21:09:41.849 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33195.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110583.log 2024-08-25T21:09:41.849 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33195.log.gz 2024-08-25T21:09:41.849 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100987.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100987.log.gz 2024-08-25T21:09:41.849 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.32093.log 2024-08-25T21:09:41.850 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110583.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110583.log.gz 2024-08-25T21:09:41.850 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119480.log 2024-08-25T21:09:41.851 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32093.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32093.log.gz 2024-08-25T21:09:41.851 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106147.log 2024-08-25T21:09:41.852 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119480.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119480.log.gz 2024-08-25T21:09:41.852 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71354.log 2024-08-25T21:09:41.853 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106147.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.106147.log.gz --verbose 2024-08-25T21:09:41.853 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.107680.log 2024-08-25T21:09:41.854 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71354.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71354.log.gz -5 2024-08-25T21:09:41.854 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.35878.log 2024-08-25T21:09:41.854 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107680.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107680.log.gz 2024-08-25T21:09:41.854 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78903.log 2024-08-25T21:09:41.855 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35878.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26589.log 2024-08-25T21:09:41.855 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35878.log.gz 2024-08-25T21:09:41.856 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78903.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78903.log.gz 2024-08-25T21:09:41.856 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48306.log 2024-08-25T21:09:41.857 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26589.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26589.log.gz 2024-08-25T21:09:41.857 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101366.log 2024-08-25T21:09:41.857 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48306.log.gz 2024-08-25T21:09:41.858 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81091.log 2024-08-25T21:09:41.858 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101366.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101366.log.gz 2024-08-25T21:09:41.858 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62155.log 2024-08-25T21:09:41.859 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81091.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81091.log.gz 2024-08-25T21:09:41.859 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49596.log 2024-08-25T21:09:41.860 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62155.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62155.log.gz 2024-08-25T21:09:41.860 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41012.log 2024-08-25T21:09:41.861 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49596.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49596.log.gz 2024-08-25T21:09:41.861 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47523.log 2024-08-25T21:09:41.862 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41012.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.41012.log.gz --verbose 2024-08-25T21:09:41.862 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.99349.log 2024-08-25T21:09:41.862 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47523.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47523.log.gz 2024-08-25T21:09:41.862 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141490.log 2024-08-25T21:09:41.863 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99349.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99349.log.gz 2024-08-25T21:09:41.863 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73011.log 2024-08-25T21:09:41.864 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141490.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141490.log.gz 2024-08-25T21:09:41.864 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139383.log 2024-08-25T21:09:41.865 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73011.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.73011.log.gz 2024-08-25T21:09:41.865 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.34142.log 2024-08-25T21:09:41.866 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139383.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.admin.139383.log.gz -- 2024-08-25T21:09:41.866 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.admin.47627.log 2024-08-25T21:09:41.866 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34142.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34142.log.gz 2024-08-25T21:09:41.866 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129926.log 2024-08-25T21:09:41.867 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.47627.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47627.log.gz 2024-08-25T21:09:41.867 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29191.log 2024-08-25T21:09:41.868 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129926.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129926.log.gz 2024-08-25T21:09:41.868 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122784.log 2024-08-25T21:09:41.869 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29191.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29191.log.gz 2024-08-25T21:09:41.869 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59710.log 2024-08-25T21:09:41.870 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122784.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122784.log.gz 2024-08-25T21:09:41.870 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125764.log 2024-08-25T21:09:41.870 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59710.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59710.log.gz 2024-08-25T21:09:41.870 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123581.log 2024-08-25T21:09:41.871 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125764.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125764.log.gz 2024-08-25T21:09:41.871 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139319.log 2024-08-25T21:09:41.872 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123581.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123581.log.gz 2024-08-25T21:09:41.872 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27487.log 2024-08-25T21:09:41.873 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139319.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89782.log 2024-08-25T21:09:41.873 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139319.log.gz 2024-08-25T21:09:41.874 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27487.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126213.log 2024-08-25T21:09:41.874 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27487.log.gz 2024-08-25T21:09:41.875 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89782.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89782.log.gz 2024-08-25T21:09:41.875 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92302.log 2024-08-25T21:09:41.876 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126213.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126213.log.gz 2024-08-25T21:09:41.876 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87219.log 2024-08-25T21:09:41.876 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92302.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92302.log.gz 2024-08-25T21:09:41.877 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92668.log 2024-08-25T21:09:41.877 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87219.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87219.log.gz 2024-08-25T21:09:41.877 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64069.log 2024-08-25T21:09:41.878 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92668.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.81787.log 2024-08-25T21:09:41.878 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.92668.log.gz 2024-08-25T21:09:41.879 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64069.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64069.log.gz 2024-08-25T21:09:41.879 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78494.log 2024-08-25T21:09:41.880 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.81787.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75807.log 2024-08-25T21:09:41.880 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.81787.log.gz 2024-08-25T21:09:41.881 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78494.log.gz 2024-08-25T21:09:41.881 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73578.log 2024-08-25T21:09:41.881 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75807.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75807.log.gz 2024-08-25T21:09:41.882 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26124.log 2024-08-25T21:09:41.882 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73578.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73578.log.gz 2024-08-25T21:09:41.882 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78584.log 2024-08-25T21:09:41.883 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26124.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26124.log.gz 2024-08-25T21:09:41.883 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73392.log 2024-08-25T21:09:41.884 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78584.log.gz 2024-08-25T21:09:41.884 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90032.log 2024-08-25T21:09:41.885 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73392.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73392.log.gz 2024-08-25T21:09:41.885 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90544.log 2024-08-25T21:09:41.886 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90032.log.gz 2024-08-25T21:09:41.886 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113254.log 2024-08-25T21:09:41.887 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90544.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.35698.log -- replaced with /var/log/ceph/cluster1-client.admin.90544.log.gz 2024-08-25T21:09:41.887 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:41.887 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113254.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113254.log.gz 2024-08-25T21:09:41.888 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72589.log 2024-08-25T21:09:41.888 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35698.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35698.log.gz 2024-08-25T21:09:41.888 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117811.log 2024-08-25T21:09:41.889 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72589.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139473.log 2024-08-25T21:09:41.889 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72589.log.gz 2024-08-25T21:09:41.890 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.117811.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117811.log.gz 2024-08-25T21:09:41.890 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98677.log 2024-08-25T21:09:41.891 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139473.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139473.log.gz 2024-08-25T21:09:41.891 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24895.log 2024-08-25T21:09:41.892 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98677.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98677.log.gz 2024-08-25T21:09:41.892 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54258.log 2024-08-25T21:09:41.892 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24895.log.gz 2024-08-25T21:09:41.892 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102260.log 2024-08-25T21:09:41.893 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54258.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54258.log.gz 2024-08-25T21:09:41.894 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79327.log 2024-08-25T21:09:41.894 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102260.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102260.log.gz 2024-08-25T21:09:41.895 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95951.log 2024-08-25T21:09:41.895 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79327.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79327.log.gz 2024-08-25T21:09:41.895 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131039.log 2024-08-25T21:09:41.896 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95951.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95951.log.gz 2024-08-25T21:09:41.896 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.138177.log 2024-08-25T21:09:41.897 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131039.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59005.log 2024-08-25T21:09:41.897 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131039.log.gz 2024-08-25T21:09:41.898 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.138177.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.138177.log.gz 2024-08-25T21:09:41.898 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138967.log 2024-08-25T21:09:41.898 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59005.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59005.log.gz 2024-08-25T21:09:41.899 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116398.log 2024-08-25T21:09:41.899 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138967.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.129678.log 2024-08-25T21:09:41.899 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138967.log.gz 2024-08-25T21:09:41.900 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116398.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116398.log.gz 2024-08-25T21:09:41.900 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.21590.log 2024-08-25T21:09:41.901 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.129678.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127450.log 2024-08-25T21:09:41.901 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.129678.log.gz 2024-08-25T21:09:41.902 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.21590.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.21590.log.gz 2024-08-25T21:09:41.902 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59210.log 2024-08-25T21:09:41.902 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127450.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127450.log.gz 2024-08-25T21:09:41.903 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95887.log 2024-08-25T21:09:41.903 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59210.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59210.log.gz 2024-08-25T21:09:41.904 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94554.log 2024-08-25T21:09:41.904 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95887.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95887.log.gz 2024-08-25T21:09:41.904 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29043.log 2024-08-25T21:09:41.905 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94554.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32385.log 2024-08-25T21:09:41.905 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94554.log.gz 2024-08-25T21:09:41.906 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29043.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29043.log.gz 2024-08-25T21:09:41.906 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141429.log 2024-08-25T21:09:41.907 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32385.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32385.log.gz 2024-08-25T21:09:41.907 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63270.log 2024-08-25T21:09:41.908 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141429.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141429.log.gz 2024-08-25T21:09:41.908 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26881.log 2024-08-25T21:09:41.908 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63270.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63270.log.gz 2024-08-25T21:09:41.909 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107127.log 2024-08-25T21:09:41.909 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26881.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26881.log.gz 2024-08-25T21:09:41.910 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32284.log 2024-08-25T21:09:41.910 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107127.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26102.log 2024-08-25T21:09:41.911 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107127.log.gz 2024-08-25T21:09:41.911 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32284.log.gz 2024-08-25T21:09:41.911 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38426.log 2024-08-25T21:09:41.912 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26102.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100525.log 2024-08-25T21:09:41.912 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26102.log.gz 2024-08-25T21:09:41.912 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38426.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.98005.log 2024-08-25T21:09:41.913 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.38426.log.gz 2024-08-25T21:09:41.913 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100525.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102790.log 2024-08-25T21:09:41.913 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100525.log.gz 2024-08-25T21:09:41.914 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98005.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98005.log.gz -5 2024-08-25T21:09:41.914 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.50646.log 2024-08-25T21:09:41.915 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102790.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102790.log.gz 2024-08-25T21:09:41.915 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81720.log 2024-08-25T21:09:41.916 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50646.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50646.log.gz 2024-08-25T21:09:41.916 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72841.log 2024-08-25T21:09:41.917 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.81720.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81720.log.gz 2024-08-25T21:09:41.917 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72137.log 2024-08-25T21:09:41.917 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72841.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128927.log 2024-08-25T21:09:41.917 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72841.log.gz 2024-08-25T21:09:41.918 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72137.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72137.log.gz 2024-08-25T21:09:41.918 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30779.log 2024-08-25T21:09:41.919 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128927.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128927.log.gz 2024-08-25T21:09:41.919 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57551.log 2024-08-25T21:09:41.920 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.30779.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30779.log.gz 2024-08-25T21:09:41.920 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117583.log 2024-08-25T21:09:41.921 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57551.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57551.log.gz 2024-08-25T21:09:41.921 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127666.log 2024-08-25T21:09:41.922 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.117583.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117583.log.gz 2024-08-25T21:09:41.922 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44412.log 2024-08-25T21:09:41.923 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127666.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127666.log.gz 2024-08-25T21:09:41.923 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102055.log 2024-08-25T21:09:41.923 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.44412.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110095.log 2024-08-25T21:09:41.924 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44412.log.gz 2024-08-25T21:09:41.924 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102055.log.gz 2024-08-25T21:09:41.924 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99623.log 2024-08-25T21:09:41.925 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110095.log.gz 2024-08-25T21:09:41.925 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44514.log 2024-08-25T21:09:41.926 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99623.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99623.log.gz 2024-08-25T21:09:41.926 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52872.log 2024-08-25T21:09:41.926 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44514.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44514.log.gz 2024-08-25T21:09:41.927 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86169.log 2024-08-25T21:09:41.927 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52872.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52872.log.gz 2024-08-25T21:09:41.928 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36794.log 2024-08-25T21:09:41.928 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86169.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86169.log.gz 2024-08-25T21:09:41.928 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40760.log 2024-08-25T21:09:41.929 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36794.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36794.log.gz 2024-08-25T21:09:41.929 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97924.log 2024-08-25T21:09:41.930 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40760.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113173.log 2024-08-25T21:09:41.930 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40760.log.gz 2024-08-25T21:09:41.931 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97924.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97924.log.gz 2024-08-25T21:09:41.931 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58835.log 2024-08-25T21:09:41.931 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113173.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113173.log.gz 2024-08-25T21:09:41.932 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111763.log 2024-08-25T21:09:41.932 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.58835.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58835.log.gz 2024-08-25T21:09:41.932 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43872.log 2024-08-25T21:09:41.933 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111763.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111763.log.gz 2024-08-25T21:09:41.933 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82290.log 2024-08-25T21:09:41.934 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43872.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43872.log.gz 2024-08-25T21:09:41.934 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124294.log 2024-08-25T21:09:41.935 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82290.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82290.log.gz 2024-08-25T21:09:41.935 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65204.log 2024-08-25T21:09:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124294.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124294.log.gz 2024-08-25T21:09:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140374.log 2024-08-25T21:09:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65204.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65204.log.gz 2024-08-25T21:09:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47163.log 2024-08-25T21:09:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.140374.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140374.log.gz 2024-08-25T21:09:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87579.log 2024-08-25T21:09:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47163.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47163.log.gz 2024-08-25T21:09:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51864.log 2024-08-25T21:09:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87579.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87579.log.gz 2024-08-25T21:09:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81901.log 2024-08-25T21:09:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51864.log.gz 2024-08-25T21:09:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114323.log 2024-08-25T21:09:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.81901.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.81901.log.gz 2024-08-25T21:09:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45474.log 2024-08-25T21:09:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114323.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114323.log.gz 2024-08-25T21:09:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124192.log 2024-08-25T21:09:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45474.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45474.log.gz 2024-08-25T21:09:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68530.log 2024-08-25T21:09:41.943 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124192.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.124192.log.gz --verbose 2024-08-25T21:09:41.943 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.136211.log 2024-08-25T21:09:41.943 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68530.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68530.log.gz 2024-08-25T21:09:41.944 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131882.log 2024-08-25T21:09:41.944 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136211.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136211.log.gz 2024-08-25T21:09:41.945 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80604.log 2024-08-25T21:09:41.946 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131882.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131882.log.gz 2024-08-25T21:09:41.946 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135877.log 2024-08-25T21:09:41.946 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80604.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80604.log.gz 2024-08-25T21:09:41.947 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94688.log 2024-08-25T21:09:41.948 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135877.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135877.log.gz 2024-08-25T21:09:41.948 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94159.log 2024-08-25T21:09:41.948 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94688.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120286.log 2024-08-25T21:09:41.949 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94688.log.gz 2024-08-25T21:09:41.949 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.94159.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94159.log.gz 2024-08-25T21:09:41.949 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48195.log 2024-08-25T21:09:41.950 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120286.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120286.log.gz 2024-08-25T21:09:41.950 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134142.log 2024-08-25T21:09:41.951 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48195.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48195.log.gz 2024-08-25T21:09:41.951 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66170.log 2024-08-25T21:09:41.952 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134142.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123622.log 2024-08-25T21:09:41.952 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134142.log.gz 2024-08-25T21:09:41.952 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66170.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66170.log.gz 2024-08-25T21:09:41.953 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124712.log 2024-08-25T21:09:41.953 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123622.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95625.log 2024-08-25T21:09:41.953 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123622.log.gz 2024-08-25T21:09:41.954 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124712.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124712.log.gz 2024-08-25T21:09:41.954 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129865.log 2024-08-25T21:09:41.955 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95625.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95625.log.gz 2024-08-25T21:09:41.955 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87429.log 2024-08-25T21:09:41.956 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129865.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129865.log.gz 2024-08-25T21:09:41.956 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120498.log 2024-08-25T21:09:41.957 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87429.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87429.log.gz 2024-08-25T21:09:41.957 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134926.log 2024-08-25T21:09:41.957 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120498.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.120498.log.gz --verbose 2024-08-25T21:09:41.958 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster2-client.admin.139888.log 2024-08-25T21:09:41.958 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134926.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134926.log.gz -5 2024-08-25T21:09:41.958 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.84364.log 2024-08-25T21:09:41.959 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.139888.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.110692.log 2024-08-25T21:09:41.959 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.139888.log.gz 2024-08-25T21:09:41.960 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84364.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84364.log.gz 2024-08-25T21:09:41.960 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.36398.log 2024-08-25T21:09:41.961 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110692.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110692.log.gz 2024-08-25T21:09:41.961 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140080.log 2024-08-25T21:09:41.962 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.36398.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.36398.log.gz 2024-08-25T21:09:41.962 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94745.log 2024-08-25T21:09:41.962 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140080.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140080.log.gz 2024-08-25T21:09:41.963 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71848.log 2024-08-25T21:09:41.964 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94745.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94745.log.gz 2024-08-25T21:09:41.964 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133742.log 2024-08-25T21:09:41.964 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71848.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71848.log.gz 2024-08-25T21:09:41.964 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130755.log 2024-08-25T21:09:41.965 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133742.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133742.log.gz 2024-08-25T21:09:41.965 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124690.log 2024-08-25T21:09:41.966 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130755.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130755.log.gz 2024-08-25T21:09:41.966 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64198.log 2024-08-25T21:09:41.967 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124690.log.gz 2024-08-25T21:09:41.967 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112519.log 2024-08-25T21:09:41.968 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64198.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64198.log.gz 2024-08-25T21:09:41.968 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52998.log 2024-08-25T21:09:41.968 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112519.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128388.log 2024-08-25T21:09:41.968 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112519.log.gz 2024-08-25T21:09:41.969 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52998.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52998.log.gz 2024-08-25T21:09:41.969 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83723.log 2024-08-25T21:09:41.970 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.128388.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128388.log.gz 2024-08-25T21:09:41.970 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83796.log 2024-08-25T21:09:41.971 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83723.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83723.log.gz 2024-08-25T21:09:41.971 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74879.log 2024-08-25T21:09:41.972 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83796.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83796.log.gz 2024-08-25T21:09:41.972 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102471.log 2024-08-25T21:09:41.973 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74879.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74879.log.gz 2024-08-25T21:09:41.973 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39079.log 2024-08-25T21:09:41.973 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.102471.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102471.log.gz 2024-08-25T21:09:41.973 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92157.log 2024-08-25T21:09:41.974 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39079.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39079.log.gz 2024-08-25T21:09:41.974 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122891.log 2024-08-25T21:09:41.975 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92157.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92157.log.gzgzip 2024-08-25T21:09:41.975 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134445.log 2024-08-25T21:09:41.976 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122891.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122891.log.gz 2024-08-25T21:09:41.976 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89929.log 2024-08-25T21:09:41.976 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134445.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134445.log.gz 2024-08-25T21:09:41.977 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.60022.log 2024-08-25T21:09:41.977 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89929.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89929.log.gz 2024-08-25T21:09:41.977 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111216.log 2024-08-25T21:09:41.978 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.60022.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.60022.log.gz 2024-08-25T21:09:41.979 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54407.log 2024-08-25T21:09:41.979 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111216.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111216.log.gz 2024-08-25T21:09:41.979 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127870.log 2024-08-25T21:09:41.980 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54407.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54407.log.gz 2024-08-25T21:09:41.980 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94460.log 2024-08-25T21:09:41.981 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.127870.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127870.log.gz 2024-08-25T21:09:41.981 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141403.log 2024-08-25T21:09:41.982 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94460.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94460.log.gz 2024-08-25T21:09:41.982 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111762.log 2024-08-25T21:09:41.982 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141403.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141403.log.gz 2024-08-25T21:09:41.983 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28437.log 2024-08-25T21:09:41.983 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111762.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111762.log.gzgzip 2024-08-25T21:09:41.983 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106542.log 2024-08-25T21:09:41.984 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28437.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28437.log.gz 2024-08-25T21:09:41.984 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66255.log 2024-08-25T21:09:41.985 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.106542.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106542.log.gz 2024-08-25T21:09:41.985 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56434.log 2024-08-25T21:09:41.986 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66255.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.66255.log.gz 2024-08-25T21:09:41.986 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95470.log 2024-08-25T21:09:41.986 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56434.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56434.log.gz 2024-08-25T21:09:41.987 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43170.log 2024-08-25T21:09:41.987 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95470.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95470.log.gz 2024-08-25T21:09:41.987 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100065.log 2024-08-25T21:09:41.988 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43170.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.43170.log.gz --verbose 2024-08-25T21:09:41.988 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.admin.93817.log 2024-08-25T21:09:41.989 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100065.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84532.log 2024-08-25T21:09:41.989 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100065.log.gz 2024-08-25T21:09:41.989 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.93817.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93817.log.gz 2024-08-25T21:09:41.989 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62911.log 2024-08-25T21:09:41.990 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84532.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84532.log.gz 2024-08-25T21:09:41.990 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75615.log 2024-08-25T21:09:41.991 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62911.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.62911.log.gz -5 2024-08-25T21:09:41.991 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.80830.log 2024-08-25T21:09:41.992 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75615.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75615.log.gz 2024-08-25T21:09:41.992 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46769.log 2024-08-25T21:09:41.992 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.80830.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80830.log.gz 2024-08-25T21:09:41.993 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31983.log 2024-08-25T21:09:41.993 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.46769.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46769.log.gz 2024-08-25T21:09:41.993 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62683.log 2024-08-25T21:09:41.994 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31983.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.131664.log 2024-08-25T21:09:41.994 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31983.log.gz 2024-08-25T21:09:41.995 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62683.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62683.log.gz 2024-08-25T21:09:41.995 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31939.log 2024-08-25T21:09:41.995 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.131664.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.131664.log.gz 2024-08-25T21:09:41.996 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112896.log 2024-08-25T21:09:41.996 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31939.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31939.log.gz 2024-08-25T21:09:41.996 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128400.log 2024-08-25T21:09:41.997 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112896.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112896.log.gz 2024-08-25T21:09:41.997 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104077.log 2024-08-25T21:09:41.998 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128400.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128400.log.gz 2024-08-25T21:09:41.998 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94232.log 2024-08-25T21:09:41.999 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104077.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104077.log.gz 2024-08-25T21:09:41.999 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75473.log 2024-08-25T21:09:41.999 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94232.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.94232.log.gz 2024-08-25T21:09:41.999 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.45404.log 2024-08-25T21:09:42.000 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75473.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75473.log.gz 2024-08-25T21:09:42.000 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130887.log 2024-08-25T21:09:42.001 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45404.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45404.log.gz 2024-08-25T21:09:42.001 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32465.log 2024-08-25T21:09:42.002 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130887.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130887.log.gz 2024-08-25T21:09:42.002 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127009.log 2024-08-25T21:09:42.003 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32465.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32465.log.gz 2024-08-25T21:09:42.003 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67320.log 2024-08-25T21:09:42.004 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127009.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127009.log.gz 2024-08-25T21:09:42.004 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119389.log 2024-08-25T21:09:42.004 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67320.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67320.log.gz 2024-08-25T21:09:42.005 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92968.log 2024-08-25T21:09:42.005 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119389.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119389.log.gz 2024-08-25T21:09:42.005 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93589.log 2024-08-25T21:09:42.006 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92968.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92968.log.gz 2024-08-25T21:09:42.006 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.143053.log 2024-08-25T21:09:42.007 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.93589.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93589.log.gz 2024-08-25T21:09:42.007 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50144.log 2024-08-25T21:09:42.008 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.143053.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.143053.log.gz 2024-08-25T21:09:42.008 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37841.log 2024-08-25T21:09:42.008 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50144.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50144.log.gz -5 2024-08-25T21:09:42.008 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.56218.log 2024-08-25T21:09:42.009 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37841.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37841.log.gz 2024-08-25T21:09:42.009 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127077.log 2024-08-25T21:09:42.010 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56218.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36156.log 0.0% 2024-08-25T21:09:42.010 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.56218.log.gz 2024-08-25T21:09:42.011 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127077.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127077.log.gz 2024-08-25T21:09:42.011 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55597.log 2024-08-25T21:09:42.011 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36156.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36156.log.gz 2024-08-25T21:09:42.011 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96660.log 2024-08-25T21:09:42.012 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55597.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55597.log.gz 2024-08-25T21:09:42.012 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111804.log 2024-08-25T21:09:42.013 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96660.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.96660.log.gz 2024-08-25T21:09:42.013 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88154.log 2024-08-25T21:09:42.013 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111804.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111804.log.gz 2024-08-25T21:09:42.014 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62725.log 2024-08-25T21:09:42.014 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88154.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88154.log.gz 2024-08-25T21:09:42.014 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137677.log 2024-08-25T21:09:42.015 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62725.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62725.log.gz 2024-08-25T21:09:42.015 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131366.log 2024-08-25T21:09:42.016 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137677.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137677.log.gz 2024-08-25T21:09:42.016 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26533.log 2024-08-25T21:09:42.016 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131366.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131366.log.gz 2024-08-25T21:09:42.016 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113003.log 2024-08-25T21:09:42.017 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26533.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26533.log.gz 2024-08-25T21:09:42.017 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129356.log 2024-08-25T21:09:42.018 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113003.log.gz 2024-08-25T21:09:42.018 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61758.log 2024-08-25T21:09:42.019 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129356.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72049.log 2024-08-25T21:09:42.019 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129356.log.gz 2024-08-25T21:09:42.019 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61758.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61758.log.gz 2024-08-25T21:09:42.020 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58968.log 2024-08-25T21:09:42.020 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72049.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72049.log.gz 2024-08-25T21:09:42.021 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.119280.log 2024-08-25T21:09:42.021 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.58968.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58968.log.gz 2024-08-25T21:09:42.021 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.119128.log 2024-08-25T21:09:42.022 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.119280.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.119280.log.gz 2024-08-25T21:09:42.022 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132942.log 2024-08-25T21:09:42.023 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.119128.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37735.log 2024-08-25T21:09:42.023 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.119128.log.gz 2024-08-25T21:09:42.024 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132942.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132942.log.gz 2024-08-25T21:09:42.024 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110388.log 2024-08-25T21:09:42.024 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37735.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.37735.log.gz 2024-08-25T21:09:42.024 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113172.log 2024-08-25T21:09:42.025 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110388.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110388.log.gz 2024-08-25T21:09:42.025 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73773.log 2024-08-25T21:09:42.026 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113172.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113172.log.gz 2024-08-25T21:09:42.026 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105972.log 2024-08-25T21:09:42.027 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73773.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73773.log.gz 2024-08-25T21:09:42.027 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64909.log 2024-08-25T21:09:42.028 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105972.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105972.log.gz 2024-08-25T21:09:42.028 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132230.log 2024-08-25T21:09:42.028 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64909.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64909.log.gz 2024-08-25T21:09:42.029 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45146.log 2024-08-25T21:09:42.030 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132230.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132230.log.gz 2024-08-25T21:09:42.030 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118308.log 2024-08-25T21:09:42.031 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45146.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45146.log.gz 2024-08-25T21:09:42.031 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61464.log 2024-08-25T21:09:42.032 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118308.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118308.log.gz 2024-08-25T21:09:42.032 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27640.log 2024-08-25T21:09:42.032 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61464.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61464.log.gz 2024-08-25T21:09:42.033 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90678.log 2024-08-25T21:09:42.033 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27640.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27640.log.gz 2024-08-25T21:09:42.034 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37460.log 2024-08-25T21:09:42.034 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90678.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90678.log.gz 2024-08-25T21:09:42.034 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51026.log 2024-08-25T21:09:42.035 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37460.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37460.log.gz 2024-08-25T21:09:42.035 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104390.log 2024-08-25T21:09:42.036 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51026.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51026.log.gz 2024-08-25T21:09:42.036 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45774.log 2024-08-25T21:09:42.037 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.104390.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104390.log.gz 2024-08-25T21:09:42.037 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118287.log 2024-08-25T21:09:42.038 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45774.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45774.log.gz 2024-08-25T21:09:42.038 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90580.log 2024-08-25T21:09:42.039 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.118287.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118287.log.gz 2024-08-25T21:09:42.039 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59690.log 2024-08-25T21:09:42.039 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.90580.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90580.log.gz 2024-08-25T21:09:42.039 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41095.log 2024-08-25T21:09:42.040 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59690.log.gz 2024-08-25T21:09:42.040 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126312.log 2024-08-25T21:09:42.041 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41095.log.gz 2024-08-25T21:09:42.041 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59625.log 2024-08-25T21:09:42.042 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126312.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126312.log.gz 2024-08-25T21:09:42.042 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30839.log 2024-08-25T21:09:42.042 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59625.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59625.log.gz 2024-08-25T21:09:42.043 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54649.log 2024-08-25T21:09:42.043 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.30839.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30839.log.gz 2024-08-25T21:09:42.043 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121291.log 2024-08-25T21:09:42.044 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54649.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54649.log.gz 2024-08-25T21:09:42.044 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113443.log 2024-08-25T21:09:42.045 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121291.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121291.log.gz 2024-08-25T21:09:42.045 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140807.log 2024-08-25T21:09:42.045 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113443.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113443.log.gz 2024-08-25T21:09:42.046 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31330.log 2024-08-25T21:09:42.046 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140807.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140807.log.gz 2024-08-25T21:09:42.046 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26189.log 2024-08-25T21:09:42.047 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31330.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31330.log.gz 2024-08-25T21:09:42.047 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.21806.log 2024-08-25T21:09:42.048 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26189.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.26189.log.gz 2024-08-25T21:09:42.048 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.24259.log 2024-08-25T21:09:42.049 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.21806.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.21806.log.gz 2024-08-25T21:09:42.049 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133781.log 2024-08-25T21:09:42.050 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24259.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24259.log.gz 2024-08-25T21:09:42.050 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74211.log 2024-08-25T21:09:42.050 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133781.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133781.log.gz 2024-08-25T21:09:42.050 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105539.log 2024-08-25T21:09:42.051 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74211.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74211.log.gz 2024-08-25T21:09:42.051 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136336.log 2024-08-25T21:09:42.052 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105539.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105539.log.gz 2024-08-25T21:09:42.052 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36507.log 2024-08-25T21:09:42.053 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136336.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136336.log.gz 2024-08-25T21:09:42.053 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136751.log 2024-08-25T21:09:42.054 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36507.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36507.log.gz 2024-08-25T21:09:42.054 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61506.log 2024-08-25T21:09:42.054 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136751.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136751.log.gz 2024-08-25T21:09:42.055 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.79981.log 2024-08-25T21:09:42.055 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61506.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61506.log.gz 2024-08-25T21:09:42.055 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93533.log 2024-08-25T21:09:42.056 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.79981.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.admin.79981.log.gz 2024-08-25T21:09:42.056 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109319.log 2024-08-25T21:09:42.057 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93533.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93533.log.gz 2024-08-25T21:09:42.057 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34666.log 2024-08-25T21:09:42.058 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109319.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109319.log.gz 2024-08-25T21:09:42.058 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40592.log 2024-08-25T21:09:42.058 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.34666.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34666.log.gz 2024-08-25T21:09:42.059 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65748.log 2024-08-25T21:09:42.059 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40592.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40592.log.gz 2024-08-25T21:09:42.059 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102470.log 2024-08-25T21:09:42.060 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65748.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65748.log.gz 2024-08-25T21:09:42.060 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56327.log 2024-08-25T21:09:42.061 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102470.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102470.log.gz 2024-08-25T21:09:42.061 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72695.log 2024-08-25T21:09:42.061 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56327.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56327.log.gz 2024-08-25T21:09:42.062 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125975.log 2024-08-25T21:09:42.062 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72695.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72695.log.gz 2024-08-25T21:09:42.063 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61276.log 2024-08-25T21:09:42.063 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125975.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125975.log.gz 2024-08-25T21:09:42.063 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40088.log 2024-08-25T21:09:42.064 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61276.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61276.log.gz 2024-08-25T21:09:42.064 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27975.log 2024-08-25T21:09:42.065 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40088.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40088.log.gz 2024-08-25T21:09:42.065 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124867.log 2024-08-25T21:09:42.065 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27975.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27975.log.gz 2024-08-25T21:09:42.066 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85916.log 2024-08-25T21:09:42.066 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124867.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124867.log.gz 2024-08-25T21:09:42.066 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126257.log 2024-08-25T21:09:42.067 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85916.log.gz 2024-08-25T21:09:42.067 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36030.log 2024-08-25T21:09:42.068 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.126257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126257.log.gz 2024-08-25T21:09:42.068 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123207.log 2024-08-25T21:09:42.069 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36030.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36030.log.gz 2024-08-25T21:09:42.069 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31397.log 2024-08-25T21:09:42.070 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123207.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123207.log.gz 2024-08-25T21:09:42.070 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70599.log 2024-08-25T21:09:42.071 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31397.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31397.log.gz 2024-08-25T21:09:42.071 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137534.log 2024-08-25T21:09:42.071 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70599.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70599.log.gz 2024-08-25T21:09:42.072 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36689.log 2024-08-25T21:09:42.072 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137534.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.137534.log.gz 2024-08-25T21:09:42.072 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.33735.log 2024-08-25T21:09:42.073 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36689.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36689.log.gz 2024-08-25T21:09:42.073 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43696.log 2024-08-25T21:09:42.074 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33735.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33735.log.gz 2024-08-25T21:09:42.074 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31611.log 2024-08-25T21:09:42.075 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43696.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43696.log.gz 2024-08-25T21:09:42.075 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84890.log 2024-08-25T21:09:42.075 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31611.log.gz 2024-08-25T21:09:42.076 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62514.log 2024-08-25T21:09:42.076 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84890.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84890.log.gz 2024-08-25T21:09:42.076 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33824.log 2024-08-25T21:09:42.077 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62514.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62514.log.gz 2024-08-25T21:09:42.077 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121793.log 2024-08-25T21:09:42.078 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33824.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33824.log.gz 2024-08-25T21:09:42.078 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48653.log 2024-08-25T21:09:42.079 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121793.log.gz 2024-08-25T21:09:42.079 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110515.log 2024-08-25T21:09:42.079 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48653.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48653.log.gz 2024-08-25T21:09:42.079 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.119433.log 2024-08-25T21:09:42.080 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110515.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110515.log.gz 2024-08-25T21:09:42.080 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67912.log 2024-08-25T21:09:42.081 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.119433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.119433.log.gz 2024-08-25T21:09:42.081 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.31408.log 2024-08-25T21:09:42.082 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67912.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67912.log.gz 2024-08-25T21:09:42.082 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39292.log 2024-08-25T21:09:42.083 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.31408.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.31408.log.gz 2024-08-25T21:09:42.083 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75346.log 2024-08-25T21:09:42.083 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39292.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39292.log.gz 2024-08-25T21:09:42.083 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125639.log 2024-08-25T21:09:42.084 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75346.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75346.log.gz 2024-08-25T21:09:42.084 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69760.log 2024-08-25T21:09:42.085 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125639.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125639.log.gz 2024-08-25T21:09:42.085 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.24896.log 2024-08-25T21:09:42.086 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.69760.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34645.log 2024-08-25T21:09:42.086 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69760.log.gz 2024-08-25T21:09:42.087 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.24896.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.24896.log.gz 2024-08-25T21:09:42.087 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84189.log 2024-08-25T21:09:42.088 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34645.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34645.log.gz 2024-08-25T21:09:42.088 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70692.log 2024-08-25T21:09:42.088 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84189.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84189.log.gz 2024-08-25T21:09:42.088 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86464.log 2024-08-25T21:09:42.089 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.70692.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70692.log.gz 2024-08-25T21:09:42.089 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119080.log 2024-08-25T21:09:42.090 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86464.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86464.log.gz 2024-08-25T21:09:42.090 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.24483.log 2024-08-25T21:09:42.091 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119080.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119080.log.gz 2024-08-25T21:09:42.091 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101658.log 2024-08-25T21:09:42.091 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.24483.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.24483.log.gz 2024-08-25T21:09:42.091 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112080.log 2024-08-25T21:09:42.092 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101658.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101658.log.gz 2024-08-25T21:09:42.092 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100735.log 2024-08-25T21:09:42.093 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112080.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112080.log.gz 2024-08-25T21:09:42.093 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64489.log 2024-08-25T21:09:42.094 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100735.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132083.log 2024-08-25T21:09:42.094 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100735.log.gz 2024-08-25T21:09:42.094 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64489.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64489.log.gz 2024-08-25T21:09:42.095 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133620.log 2024-08-25T21:09:42.095 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132083.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132083.log.gz 2024-08-25T21:09:42.095 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83207.log 2024-08-25T21:09:42.096 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133620.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133620.log.gz 2024-08-25T21:09:42.096 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120285.log 2024-08-25T21:09:42.097 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83207.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74715.log 2024-08-25T21:09:42.097 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83207.log.gz 2024-08-25T21:09:42.098 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120285.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120285.log.gz 2024-08-25T21:09:42.098 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137784.log 2024-08-25T21:09:42.098 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74715.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74715.log.gz 2024-08-25T21:09:42.099 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129269.log 2024-08-25T21:09:42.099 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137784.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114378.log 2024-08-25T21:09:42.099 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137784.log.gz 2024-08-25T21:09:42.100 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129269.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129269.log.gz 2024-08-25T21:09:42.100 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27111.log 2024-08-25T21:09:42.101 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114378.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114378.log.gz 2024-08-25T21:09:42.101 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112056.log 2024-08-25T21:09:42.102 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27111.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27111.log.gz 2024-08-25T21:09:42.102 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103342.log 2024-08-25T21:09:42.102 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112056.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112056.log.gz 2024-08-25T21:09:42.103 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48548.log 2024-08-25T21:09:42.103 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103342.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103342.log.gz 2024-08-25T21:09:42.103 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78184.log 2024-08-25T21:09:42.104 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48548.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48548.log.gz 2024-08-25T21:09:42.104 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83421.log 2024-08-25T21:09:42.105 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78184.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112478.log 2024-08-25T21:09:42.105 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78184.log.gz 2024-08-25T21:09:42.106 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83421.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83421.log.gz 2024-08-25T21:09:42.106 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137999.log 2024-08-25T21:09:42.106 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112478.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112478.log.gz 2024-08-25T21:09:42.107 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82699.log 2024-08-25T21:09:42.107 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.137999.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137999.log.gz 2024-08-25T21:09:42.108 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97875.log 2024-08-25T21:09:42.108 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82699.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82699.log.gz 2024-08-25T21:09:42.108 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31875.log 2024-08-25T21:09:42.109 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97875.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97875.log.gz 2024-08-25T21:09:42.109 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85497.log 2024-08-25T21:09:42.110 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31875.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31875.log.gz 2024-08-25T21:09:42.110 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110672.log 2024-08-25T21:09:42.111 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85497.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137533.log 2024-08-25T21:09:42.111 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85497.log.gz 2024-08-25T21:09:42.111 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110672.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110672.log.gz 2024-08-25T21:09:42.112 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83644.log 2024-08-25T21:09:42.112 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137533.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137533.log.gz 2024-08-25T21:09:42.112 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60609.log 2024-08-25T21:09:42.113 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83644.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125643.log 2024-08-25T21:09:42.113 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83644.log.gz 2024-08-25T21:09:42.114 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60609.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60609.log.gz 2024-08-25T21:09:42.114 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52034.log 2024-08-25T21:09:42.115 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125643.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125643.log.gz 2024-08-25T21:09:42.115 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89959.log 2024-08-25T21:09:42.116 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52034.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52034.log.gz 2024-08-25T21:09:42.116 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135494.log 2024-08-25T21:09:42.116 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110923.log 2024-08-25T21:09:42.117 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89959.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89959.log.gz 2024-08-25T21:09:42.117 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135494.log.gz 2024-08-25T21:09:42.117 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.138333.log 2024-08-25T21:09:42.118 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110923.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110923.log.gz 2024-08-25T21:09:42.118 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112621.log 2024-08-25T21:09:42.119 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.138333.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.138333.log.gz 2024-08-25T21:09:42.119 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.131146.log 2024-08-25T21:09:42.120 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112621.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112621.log.gz 2024-08-25T21:09:42.120 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47828.log 2024-08-25T21:09:42.120 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.131146.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.131146.log.gz 2024-08-25T21:09:42.120 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107436.log 2024-08-25T21:09:42.121 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47828.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47828.log.gz 2024-08-25T21:09:42.121 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81974.log 2024-08-25T21:09:42.122 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107436.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107436.log.gz 2024-08-25T21:09:42.122 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.141019.log 2024-08-25T21:09:42.123 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.81974.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81974.log.gz 2024-08-25T21:09:42.123 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124191.log 2024-08-25T21:09:42.123 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.141019.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.141019.log.gz 2024-08-25T21:09:42.124 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34894.log 2024-08-25T21:09:42.124 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124191.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124191.log.gz 2024-08-25T21:09:42.124 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77047.log 2024-08-25T21:09:42.125 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.34894.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34894.log.gz 2024-08-25T21:09:42.126 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96974.log 2024-08-25T21:09:42.126 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102525.log 2024-08-25T21:09:42.127 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77047.log.gz 2024-08-25T21:09:42.127 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96974.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96974.log.gz 2024-08-25T21:09:42.127 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49640.log 2024-08-25T21:09:42.128 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102525.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102525.log.gz 2024-08-25T21:09:42.128 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119738.log 2024-08-25T21:09:42.129 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49640.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.mirror.49640.log.gz /var/log/ceph/cluster1-client.mirror.37987.log 2024-08-25T21:09:42.129 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:42.130 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119738.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108878.log 2024-08-25T21:09:42.130 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119738.log.gz 2024-08-25T21:09:42.130 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37987.log.gz 2024-08-25T21:09:42.130 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138722.log 2024-08-25T21:09:42.131 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108878.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122593.log 2024-08-25T21:09:42.131 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108878.log.gz 2024-08-25T21:09:42.132 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.138722.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138722.log.gz 2024-08-25T21:09:42.132 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95164.log 2024-08-25T21:09:42.133 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122593.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122593.log.gz 2024-08-25T21:09:42.133 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71695.log 2024-08-25T21:09:42.133 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95164.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95164.log.gz 2024-08-25T21:09:42.134 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23423.log 2024-08-25T21:09:42.135 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71695.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71695.log.gz 2024-08-25T21:09:42.135 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56544.log 2024-08-25T21:09:42.135 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23423.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.23423.log.gz 2024-08-25T21:09:42.135 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122093.log 2024-08-25T21:09:42.136 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56544.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56544.log.gz 2024-08-25T21:09:42.136 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77070.log 2024-08-25T21:09:42.137 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122093.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122093.log.gz 2024-08-25T21:09:42.137 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72422.log 2024-08-25T21:09:42.138 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77070.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77070.log.gz 2024-08-25T21:09:42.138 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115452.log 2024-08-25T21:09:42.139 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72422.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72422.log.gz 2024-08-25T21:09:42.139 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129470.log 2024-08-25T21:09:42.140 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115452.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115452.log.gz 2024-08-25T21:09:42.140 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75952.log 2024-08-25T21:09:42.141 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129470.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129470.log.gz 2024-08-25T21:09:42.141 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124236.log 2024-08-25T21:09:42.141 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75952.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75952.log.gz 2024-08-25T21:09:42.142 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54959.log 2024-08-25T21:09:42.142 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124236.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124236.log.gz 2024-08-25T21:09:42.143 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69407.log 2024-08-25T21:09:42.143 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54959.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54959.log.gz 2024-08-25T21:09:42.143 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27689.log 2024-08-25T21:09:42.144 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.69407.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69407.log.gz 2024-08-25T21:09:42.144 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62220.log 2024-08-25T21:09:42.145 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27689.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27689.log.gz 2024-08-25T21:09:42.145 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37653.log 2024-08-25T21:09:42.146 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62220.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62220.log.gz 2024-08-25T21:09:42.146 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99915.log 2024-08-25T21:09:42.147 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37653.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37653.log.gz 2024-08-25T21:09:42.147 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139363.log 2024-08-25T21:09:42.148 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99915.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99915.log.gz 2024-08-25T21:09:42.148 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46728.log 2024-08-25T21:09:42.149 INFO:teuthology.orchestra.run.smithi119.stderr: 91.6% -- replaced with /var/log/ceph/cluster1-mgr.x.log.gz 2024-08-25T21:09:42.149 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139363.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139363.log.gz 2024-08-25T21:09:42.149 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53628.log 2024-08-25T21:09:42.150 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46728.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.46728.log.gz 2024-08-25T21:09:42.150 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37925.log 2024-08-25T21:09:42.151 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53628.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53628.log.gz 2024-08-25T21:09:42.151 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26943.log 2024-08-25T21:09:42.151 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37925.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31587.log 2024-08-25T21:09:42.152 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37925.log.gz 2024-08-25T21:09:42.152 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26943.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26943.log.gz 2024-08-25T21:09:42.152 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55839.log 2024-08-25T21:09:42.153 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31587.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31587.log.gz 2024-08-25T21:09:42.153 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109171.log 2024-08-25T21:09:42.154 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55839.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55839.log.gz 2024-08-25T21:09:42.154 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108207.log 2024-08-25T21:09:42.155 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109171.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109171.log.gz 2024-08-25T21:09:42.155 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99624.log 2024-08-25T21:09:42.155 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.108207.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108207.log.gz 2024-08-25T21:09:42.156 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40299.log 2024-08-25T21:09:42.156 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99624.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99624.log.gz 2024-08-25T21:09:42.157 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110414.log 2024-08-25T21:09:42.158 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40299.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40299.log.gz 2024-08-25T21:09:42.158 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54744.log 2024-08-25T21:09:42.158 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110414.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110414.log.gz 2024-08-25T21:09:42.159 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72340.log 2024-08-25T21:09:42.159 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54744.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54744.log.gz 2024-08-25T21:09:42.160 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86254.log 2024-08-25T21:09:42.160 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72340.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72340.log.gz 2024-08-25T21:09:42.160 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82832.log 2024-08-25T21:09:42.161 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86254.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86254.log.gz 2024-08-25T21:09:42.161 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41935.log 2024-08-25T21:09:42.162 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82832.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82832.log.gz 2024-08-25T21:09:42.162 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.33433.log 2024-08-25T21:09:42.163 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41935.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41935.log.gz 2024-08-25T21:09:42.163 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79504.log 2024-08-25T21:09:42.164 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.33433.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.33433.log.gz 2024-08-25T21:09:42.164 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54543.log 2024-08-25T21:09:42.164 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79504.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79504.log.gz 2024-08-25T21:09:42.165 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31675.log 2024-08-25T21:09:42.165 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54543.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54543.log.gz 2024-08-25T21:09:42.166 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92407.log 2024-08-25T21:09:42.166 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31675.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31675.log.gz 2024-08-25T21:09:42.166 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140679.log 2024-08-25T21:09:42.167 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92407.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.85833.log 2024-08-25T21:09:42.167 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.92407.log.gz 2024-08-25T21:09:42.168 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.140679.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.140679.log.gz -5 --verbose 2024-08-25T21:09:42.168 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.98678.log 2024-08-25T21:09:42.169 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85833.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85833.log.gz 2024-08-25T21:09:42.169 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73774.log 2024-08-25T21:09:42.169 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98678.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60322.log 2024-08-25T21:09:42.169 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98678.log.gz 2024-08-25T21:09:42.170 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73774.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.99434.log 2024-08-25T21:09:42.170 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.73774.log.gz 2024-08-25T21:09:42.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60322.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60322.log.gz 2024-08-25T21:09:42.171 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100794.log 2024-08-25T21:09:42.172 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99434.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99434.log.gz 2024-08-25T21:09:42.172 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86842.log 2024-08-25T21:09:42.173 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100794.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100794.log.gz 2024-08-25T21:09:42.173 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128307.log 2024-08-25T21:09:42.173 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.86842.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86842.log.gz 2024-08-25T21:09:42.174 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38366.log 2024-08-25T21:09:42.174 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128307.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128307.log.gz 2024-08-25T21:09:42.174 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66359.log 2024-08-25T21:09:42.175 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38366.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38366.log.gz 2024-08-25T21:09:42.175 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66094.log 2024-08-25T21:09:42.176 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66359.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66359.log.gz 2024-08-25T21:09:42.176 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.22993.log 2024-08-25T21:09:42.177 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66094.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66094.log.gz 2024-08-25T21:09:42.177 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85431.log 2024-08-25T21:09:42.177 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.22993.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.22993.log.gz 2024-08-25T21:09:42.177 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96953.log 2024-08-25T21:09:42.178 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85431.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134193.log 2024-08-25T21:09:42.178 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85431.log.gz 2024-08-25T21:09:42.179 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96953.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96953.log.gz 2024-08-25T21:09:42.179 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54935.log 2024-08-25T21:09:42.180 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134193.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134193.log.gz 2024-08-25T21:09:42.180 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131577.log 2024-08-25T21:09:42.180 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54935.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54935.log.gz 2024-08-25T21:09:42.181 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61968.log 2024-08-25T21:09:42.181 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131577.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131577.log.gz 2024-08-25T21:09:42.181 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133171.log 2024-08-25T21:09:42.182 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61968.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61968.log.gz 2024-08-25T21:09:42.182 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83027.log 2024-08-25T21:09:42.183 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133171.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133171.log.gz 2024-08-25T21:09:42.183 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130067.log 2024-08-25T21:09:42.184 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.83027.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83027.log.gz 2024-08-25T21:09:42.184 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76477.log 2024-08-25T21:09:42.185 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.130067.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130067.log.gz 2024-08-25T21:09:42.185 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98929.log 2024-08-25T21:09:42.186 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76477.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82129.log 2024-08-25T21:09:42.186 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76477.log.gz 2024-08-25T21:09:42.187 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98929.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98929.log.gz 2024-08-25T21:09:42.187 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69188.log 2024-08-25T21:09:42.187 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82129.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82129.log.gz 2024-08-25T21:09:42.187 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.23010.log 2024-08-25T21:09:42.188 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69188.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69188.log.gz 2024-08-25T21:09:42.188 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134034.log 2024-08-25T21:09:42.189 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.23010.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.23010.log.gz 2024-08-25T21:09:42.189 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89260.log 2024-08-25T21:09:42.189 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134034.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134034.log.gz 2024-08-25T21:09:42.190 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122134.log 2024-08-25T21:09:42.191 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89260.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89260.log.gz -5 2024-08-25T21:09:42.191 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.34686.log 2024-08-25T21:09:42.191 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122134.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122134.log.gz 2024-08-25T21:09:42.192 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116630.log 2024-08-25T21:09:42.192 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34686.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34686.log.gz 2024-08-25T21:09:42.192 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38619.log 2024-08-25T21:09:42.193 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.116630.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116630.log.gz 2024-08-25T21:09:42.193 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91798.log 2024-08-25T21:09:42.194 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38619.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38619.log.gz 2024-08-25T21:09:42.194 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54176.log 2024-08-25T21:09:42.195 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91798.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91798.log.gz 2024-08-25T21:09:42.195 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129725.log 2024-08-25T21:09:42.195 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54176.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54176.log.gz 2024-08-25T21:09:42.196 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61695.log 2024-08-25T21:09:42.196 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129725.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129725.log.gz 2024-08-25T21:09:42.196 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41643.log 2024-08-25T21:09:42.197 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61695.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61695.log.gz 2024-08-25T21:09:42.197 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47980.log 2024-08-25T21:09:42.198 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41643.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41643.log.gz 2024-08-25T21:09:42.198 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88943.log 2024-08-25T21:09:42.199 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47980.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47980.log.gz 2024-08-25T21:09:42.199 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39038.log 2024-08-25T21:09:42.199 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65560.log 2024-08-25T21:09:42.200 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88943.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88943.log.gz 2024-08-25T21:09:42.200 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39038.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39038.log.gz 2024-08-25T21:09:42.200 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139033.log 2024-08-25T21:09:42.201 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65560.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65560.log.gz 2024-08-25T21:09:42.201 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125682.log 2024-08-25T21:09:42.202 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139033.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139033.log.gz 2024-08-25T21:09:42.202 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105744.log 2024-08-25T21:09:42.203 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125682.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125682.log.gz 2024-08-25T21:09:42.203 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128664.log 2024-08-25T21:09:42.203 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105744.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105744.log.gz 2024-08-25T21:09:42.203 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41389.log 2024-08-25T21:09:42.204 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128664.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128664.log.gz 2024-08-25T21:09:42.204 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78518.log 2024-08-25T21:09:42.205 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41389.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41389.log.gz 2024-08-25T21:09:42.205 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53210.log 2024-08-25T21:09:42.206 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78518.log.gz 2024-08-25T21:09:42.206 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36733.log 2024-08-25T21:09:42.206 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53210.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73494.log 2024-08-25T21:09:42.207 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53210.log.gz 2024-08-25T21:09:42.207 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36733.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36733.log.gz 2024-08-25T21:09:42.207 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.23729.log 2024-08-25T21:09:42.208 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73494.log.gz 2024-08-25T21:09:42.208 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29591.log 2024-08-25T21:09:42.209 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.23729.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.23729.log.gz 2024-08-25T21:09:42.209 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61028.log 2024-08-25T21:09:42.210 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29591.log.gz 2024-08-25T21:09:42.210 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119479.log 2024-08-25T21:09:42.211 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61028.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61028.log.gz 2024-08-25T21:09:42.211 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112561.log 2024-08-25T21:09:42.212 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119479.log.gz 2024-08-25T21:09:42.212 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35201.log 2024-08-25T21:09:42.213 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112561.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63752.log 2024-08-25T21:09:42.213 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112561.log.gz 2024-08-25T21:09:42.213 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.35201.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35201.log.gz 2024-08-25T21:09:42.213 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43587.log 2024-08-25T21:09:42.214 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63752.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63752.log.gz 2024-08-25T21:09:42.215 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77447.log 2024-08-25T21:09:42.215 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43587.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43587.log.gz 2024-08-25T21:09:42.215 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70956.log 2024-08-25T21:09:42.216 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77447.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77447.log.gz 2024-08-25T21:09:42.216 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28479.log 2024-08-25T21:09:42.217 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70956.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.57943.log 2024-08-25T21:09:42.217 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70956.log.gz 2024-08-25T21:09:42.217 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28479.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.28479.log.gz 2024-08-25T21:09:42.218 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.121670.log 2024-08-25T21:09:42.219 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.57943.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131077.log 2024-08-25T21:09:42.219 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.57943.log.gz 2024-08-25T21:09:42.219 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121670.log.gz 2024-08-25T21:09:42.219 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98047.log 2024-08-25T21:09:42.220 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131077.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.131077.log.gz -5 --verbose 2024-08-25T21:09:42.220 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.admin.66653.log 2024-08-25T21:09:42.221 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98047.log.gz 2024-08-25T21:09:42.221 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102303.log 2024-08-25T21:09:42.222 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66653.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114365.log 2024-08-25T21:09:42.222 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66653.log.gz 2024-08-25T21:09:42.222 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102303.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102303.log.gz 2024-08-25T21:09:42.223 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129178.log 2024-08-25T21:09:42.223 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.114365.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114365.log.gz 2024-08-25T21:09:42.224 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60500.log 2024-08-25T21:09:42.224 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.129178.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129178.log.gz 2024-08-25T21:09:42.224 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32257.log 2024-08-25T21:09:42.225 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60500.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60500.log.gz 2024-08-25T21:09:42.225 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141745.log 2024-08-25T21:09:42.226 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32257.log.gz 2024-08-25T21:09:42.226 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41474.log 2024-08-25T21:09:42.227 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141745.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141745.log.gz 2024-08-25T21:09:42.227 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41314.log 2024-08-25T21:09:42.228 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41474.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41474.log.gz 2024-08-25T21:09:42.228 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79927.log 2024-08-25T21:09:42.228 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41314.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41314.log.gz 2024-08-25T21:09:42.229 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63145.log 2024-08-25T21:09:42.229 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79927.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79927.log.gz 2024-08-25T21:09:42.230 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34531.log 2024-08-25T21:09:42.230 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63145.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63145.log.gz 2024-08-25T21:09:42.231 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41644.log 2024-08-25T21:09:42.231 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.34531.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34531.log.gz 2024-08-25T21:09:42.231 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71284.log 2024-08-25T21:09:42.232 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41644.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41644.log.gz 2024-08-25T21:09:42.232 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99183.log 2024-08-25T21:09:42.233 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.71284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71284.log.gz 2024-08-25T21:09:42.233 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127667.log 2024-08-25T21:09:42.234 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99183.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99183.log.gz 2024-08-25T21:09:42.234 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107935.log 2024-08-25T21:09:42.235 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127667.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140107.log 2024-08-25T21:09:42.235 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127667.log.gz 2024-08-25T21:09:42.236 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.107935.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107935.log.gz 2024-08-25T21:09:42.236 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29696.log 2024-08-25T21:09:42.236 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.140107.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140107.log.gz 2024-08-25T21:09:42.237 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25681.log 2024-08-25T21:09:42.237 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29696.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29696.log.gz 2024-08-25T21:09:42.238 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56091.log 2024-08-25T21:09:42.238 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25681.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25681.log.gz 2024-08-25T21:09:42.239 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107611.log 2024-08-25T21:09:42.240 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.142227.log 2024-08-25T21:09:42.240 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56091.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56091.log.gz 2024-08-25T21:09:42.241 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107611.log.gz 2024-08-25T21:09:42.241 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136060.log 2024-08-25T21:09:42.242 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.142227.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.142227.log.gz 2024-08-25T21:09:42.242 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.23838.log 2024-08-25T21:09:42.242 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136060.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136060.log.gzgzip 2024-08-25T21:09:42.242 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89958.log 2024-08-25T21:09:42.243 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.23838.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster2-client.admin.33329.log -- replaced with /var/log/ceph/cluster1-client.mirror.23838.log.gz 2024-08-25T21:09:42.243 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:42.244 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89958.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89958.log.gz 2024-08-25T21:09:42.244 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101764.log 2024-08-25T21:09:42.245 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33329.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33329.log.gz 2024-08-25T21:09:42.245 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125936.log 2024-08-25T21:09:42.246 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101764.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101764.log.gz 2024-08-25T21:09:42.246 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80423.log 2024-08-25T21:09:42.247 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125936.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.125936.log.gz --verbose 2024-08-25T21:09:42.247 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.129564.log 2024-08-25T21:09:42.248 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.80423.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69643.log 2024-08-25T21:09:42.248 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80423.log.gz 2024-08-25T21:09:42.248 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.129564.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.129564.log.gz 2024-08-25T21:09:42.248 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95233.log 2024-08-25T21:09:42.249 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69643.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69643.log.gz 2024-08-25T21:09:42.250 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33963.log 2024-08-25T21:09:42.250 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95233.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95233.log.gz 2024-08-25T21:09:42.250 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50824.log 2024-08-25T21:09:42.251 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.33963.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33963.log.gz 2024-08-25T21:09:42.251 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39793.log 2024-08-25T21:09:42.252 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50824.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50824.log.gz 2024-08-25T21:09:42.252 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61548.log 2024-08-25T21:09:42.253 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39793.log.gz 2024-08-25T21:09:42.253 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75891.log 2024-08-25T21:09:42.254 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114277.log 2024-08-25T21:09:42.254 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61548.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61548.log.gz 2024-08-25T21:09:42.254 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75891.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75891.log.gz 2024-08-25T21:09:42.255 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51024.log 2024-08-25T21:09:42.255 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114277.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114277.log.gz 2024-08-25T21:09:42.255 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68228.log 2024-08-25T21:09:42.256 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51024.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49870.log 2024-08-25T21:09:42.256 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51024.log.gz 2024-08-25T21:09:42.257 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.68228.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68228.log.gz 2024-08-25T21:09:42.257 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105263.log 2024-08-25T21:09:42.257 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49870.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49870.log.gz 2024-08-25T21:09:42.258 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93250.log 2024-08-25T21:09:42.258 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105263.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105263.log.gz 2024-08-25T21:09:42.258 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104819.log 2024-08-25T21:09:42.259 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.93250.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93250.log.gz 2024-08-25T21:09:42.259 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105992.log 2024-08-25T21:09:42.260 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.104819.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104819.log.gz 2024-08-25T21:09:42.260 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84121.log 2024-08-25T21:09:42.261 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.105992.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster2-client.mirror.105992.log.gz -- 2024-08-25T21:09:42.261 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.122930.log 2024-08-25T21:09:42.262 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84121.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84121.log.gz 2024-08-25T21:09:42.262 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41811.log 2024-08-25T21:09:42.262 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122930.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122930.log.gz 2024-08-25T21:09:42.263 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77365.log 2024-08-25T21:09:42.263 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41811.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41811.log.gz 2024-08-25T21:09:42.263 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99861.log 2024-08-25T21:09:42.264 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77365.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77365.log.gz 2024-08-25T21:09:42.264 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99916.log 2024-08-25T21:09:42.265 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99861.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99861.log.gz 2024-08-25T21:09:42.265 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27721.log 2024-08-25T21:09:42.266 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99916.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124844.log 2024-08-25T21:09:42.266 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99916.log.gz 2024-08-25T21:09:42.267 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27721.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38092.log 2024-08-25T21:09:42.267 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27721.log.gz 2024-08-25T21:09:42.267 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124844.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124844.log.gz 2024-08-25T21:09:42.267 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78599.log 2024-08-25T21:09:42.268 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38092.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76790.log 2024-08-25T21:09:42.268 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38092.log.gz 2024-08-25T21:09:42.269 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78599.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78599.log.gz 2024-08-25T21:09:42.269 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119655.log 2024-08-25T21:09:42.270 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76790.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49492.log 2024-08-25T21:09:42.270 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76790.log.gz 2024-08-25T21:09:42.270 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119655.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.119655.log.gz 2024-08-25T21:09:42.271 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25567.log 2024-08-25T21:09:42.271 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49492.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49492.log.gz 2024-08-25T21:09:42.271 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135876.log 2024-08-25T21:09:42.272 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25567.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25567.log.gz 2024-08-25T21:09:42.272 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136543.log 2024-08-25T21:09:42.273 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135876.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135876.log.gz 2024-08-25T21:09:42.273 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79746.log 2024-08-25T21:09:42.274 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136543.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136543.log.gz 2024-08-25T21:09:42.274 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137050.log 2024-08-25T21:09:42.275 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79746.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79746.log.gz 2024-08-25T21:09:42.275 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121461.log 2024-08-25T21:09:42.276 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137050.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.91879.log 2024-08-25T21:09:42.276 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137050.log.gz 2024-08-25T21:09:42.276 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121461.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121461.log.gz 2024-08-25T21:09:42.277 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60321.log 2024-08-25T21:09:42.277 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.91879.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.91879.log.gz 2024-08-25T21:09:42.278 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92075.log 2024-08-25T21:09:42.278 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60321.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60321.log.gz 2024-08-25T21:09:42.279 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99013.log 2024-08-25T21:09:42.279 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92075.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92075.log.gz 2024-08-25T21:09:42.280 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140026.log 2024-08-25T21:09:42.280 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99013.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140673.log 2024-08-25T21:09:42.280 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99013.log.gz 2024-08-25T21:09:42.281 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.140026.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40847.log 2024-08-25T21:09:42.281 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140026.log.gz 2024-08-25T21:09:42.282 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.140673.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140673.log.gz 2024-08-25T21:09:42.282 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109801.log 2024-08-25T21:09:42.283 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40847.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40847.log.gz 2024-08-25T21:09:42.283 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66637.log 2024-08-25T21:09:42.283 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109801.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92142.log 2024-08-25T21:09:42.283 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109801.log.gz 2024-08-25T21:09:42.284 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66637.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66637.log.gz 2024-08-25T21:09:42.284 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83833.log 2024-08-25T21:09:42.285 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92142.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92142.log.gz 2024-08-25T21:09:42.285 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63816.log 2024-08-25T21:09:42.286 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83833.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72339.log 2024-08-25T21:09:42.286 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83833.log.gz 2024-08-25T21:09:42.286 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63816.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63816.log.gz 2024-08-25T21:09:42.286 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98445.log 2024-08-25T21:09:42.287 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72339.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61947.log 2024-08-25T21:09:42.287 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72339.log.gz 2024-08-25T21:09:42.288 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98445.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98445.log.gz 2024-08-25T21:09:42.288 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100170.log 2024-08-25T21:09:42.289 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61947.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61947.log.gz 2024-08-25T21:09:42.289 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52957.log 2024-08-25T21:09:42.290 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100170.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100170.log.gz 2024-08-25T21:09:42.290 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30967.log 2024-08-25T21:09:42.290 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52957.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52957.log.gz 2024-08-25T21:09:42.291 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43461.log 2024-08-25T21:09:42.291 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.30967.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66789.log 0.0% 2024-08-25T21:09:42.291 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.30967.log.gz 2024-08-25T21:09:42.292 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.43461.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43461.log.gz 2024-08-25T21:09:42.292 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39754.log 2024-08-25T21:09:42.293 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66789.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66789.log.gz 2024-08-25T21:09:42.293 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74149.log 2024-08-25T21:09:42.294 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39754.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39754.log.gz 2024-08-25T21:09:42.294 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92591.log 2024-08-25T21:09:42.295 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74149.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74149.log.gz 2024-08-25T21:09:42.295 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37712.log 2024-08-25T21:09:42.296 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92591.log.gz 2024-08-25T21:09:42.296 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124505.log 2024-08-25T21:09:42.297 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37712.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37712.log.gz 2024-08-25T21:09:42.297 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74632.log 2024-08-25T21:09:42.298 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124505.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124505.log.gz 2024-08-25T21:09:42.298 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.115927.log 2024-08-25T21:09:42.299 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74632.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74632.log.gz 2024-08-25T21:09:42.299 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124885.log 2024-08-25T21:09:42.299 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.115927.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.115927.log.gz 2024-08-25T21:09:42.300 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30160.log 2024-08-25T21:09:42.301 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124885.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124885.log.gz 2024-08-25T21:09:42.301 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61089.log 2024-08-25T21:09:42.301 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30160.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54768.log 2024-08-25T21:09:42.302 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30160.log.gz 2024-08-25T21:09:42.302 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61089.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61089.log.gz 2024-08-25T21:09:42.302 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141771.log 2024-08-25T21:09:42.303 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54768.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54768.log.gz 2024-08-25T21:09:42.303 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58057.log 2024-08-25T21:09:42.304 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.141771.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141771.log.gz 2024-08-25T21:09:42.304 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65938.log 2024-08-25T21:09:42.305 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58057.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58057.log.gz 2024-08-25T21:09:42.305 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129295.log 2024-08-25T21:09:42.305 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65938.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65938.log.gz 2024-08-25T21:09:42.306 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42021.log 2024-08-25T21:09:42.306 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129295.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129295.log.gz 2024-08-25T21:09:42.306 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73370.log 2024-08-25T21:09:42.307 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42021.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42021.log.gz 2024-08-25T21:09:42.307 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104737.log 2024-08-25T21:09:42.308 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73370.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73370.log.gz 2024-08-25T21:09:42.308 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.32497.log 2024-08-25T21:09:42.309 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104737.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104737.log.gz 2024-08-25T21:09:42.309 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128228.log 2024-08-25T21:09:42.309 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.32497.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32497.log.gz 2024-08-25T21:09:42.310 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54782.log 2024-08-25T21:09:42.310 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128228.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103230.log 2024-08-25T21:09:42.310 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128228.log.gz 2024-08-25T21:09:42.311 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54782.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54782.log.gz 2024-08-25T21:09:42.311 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97585.log 2024-08-25T21:09:42.312 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103230.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103230.log.gz 2024-08-25T21:09:42.312 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64113.log 2024-08-25T21:09:42.313 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97585.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97585.log.gz 2024-08-25T21:09:42.313 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60752.log 2024-08-25T21:09:42.313 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64113.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64113.log.gz 2024-08-25T21:09:42.313 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133884.log 2024-08-25T21:09:42.314 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60752.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60752.log.gz 2024-08-25T21:09:42.314 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22056.log 2024-08-25T21:09:42.315 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.133884.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133884.log.gz 2024-08-25T21:09:42.315 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134296.log 2024-08-25T21:09:42.316 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22056.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22056.log.gz 2024-08-25T21:09:42.316 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48761.log 2024-08-25T21:09:42.317 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134296.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134296.log.gz 2024-08-25T21:09:42.317 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136545.log 2024-08-25T21:09:42.318 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48761.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48761.log.gz 2024-08-25T21:09:42.318 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60086.log 2024-08-25T21:09:42.318 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136545.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136545.log.gz 2024-08-25T21:09:42.319 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65053.log 2024-08-25T21:09:42.319 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60086.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77210.log 2024-08-25T21:09:42.319 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60086.log.gz 2024-08-25T21:09:42.320 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65053.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118728.log 2024-08-25T21:09:42.320 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65053.log.gz 2024-08-25T21:09:42.321 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77210.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.142633.log 2024-08-25T21:09:42.321 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77210.log.gz 2024-08-25T21:09:42.321 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118728.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118728.log.gz 2024-08-25T21:09:42.322 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121671.log 2024-08-25T21:09:42.322 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.142633.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.admin.142633.log.gz -5 2024-08-25T21:09:42.322 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.48862.log 2024-08-25T21:09:42.323 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121671.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121671.log.gz 2024-08-25T21:09:42.323 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52370.log 2024-08-25T21:09:42.324 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48862.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122932.log 2024-08-25T21:09:42.324 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48862.log.gz 2024-08-25T21:09:42.325 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52370.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52370.log.gz 2024-08-25T21:09:42.325 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59931.log 2024-08-25T21:09:42.326 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122932.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79571.log 2024-08-25T21:09:42.326 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122932.log.gz 2024-08-25T21:09:42.326 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59931.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59931.log.gz 2024-08-25T21:09:42.326 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67756.log 2024-08-25T21:09:42.327 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79571.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63069.log 2024-08-25T21:09:42.327 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79571.log.gz 2024-08-25T21:09:42.328 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67756.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58426.log 0.0% 2024-08-25T21:09:42.328 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.67756.log.gz 2024-08-25T21:09:42.329 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63069.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41097.log 2024-08-25T21:09:42.329 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63069.log.gz 2024-08-25T21:09:42.329 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58426.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58426.log.gz 2024-08-25T21:09:42.330 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129979.log 2024-08-25T21:09:42.330 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41097.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41097.log.gz 2024-08-25T21:09:42.330 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26715.log 2024-08-25T21:09:42.331 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129979.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129979.log.gz 2024-08-25T21:09:42.331 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34344.log 2024-08-25T21:09:42.332 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26715.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30911.log 2024-08-25T21:09:42.332 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26715.log.gz 2024-08-25T21:09:42.333 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.34344.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34344.log.gz 2024-08-25T21:09:42.333 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58312.log 2024-08-25T21:09:42.333 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30911.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30911.log.gz 2024-08-25T21:09:42.333 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44178.log 2024-08-25T21:09:42.334 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58312.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58312.log.gz 2024-08-25T21:09:42.334 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.59027.log 2024-08-25T21:09:42.335 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44178.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44178.log.gz 2024-08-25T21:09:42.335 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104821.log 2024-08-25T21:09:42.336 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.59027.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59027.log.gz 2024-08-25T21:09:42.336 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132441.log 2024-08-25T21:09:42.336 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104821.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104821.log.gz 2024-08-25T21:09:42.336 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65370.log 2024-08-25T21:09:42.337 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132441.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132441.log.gz 2024-08-25T21:09:42.337 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57169.log 2024-08-25T21:09:42.338 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65370.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.65370.log.gz --verbose 2024-08-25T21:09:42.338 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.27135.log 2024-08-25T21:09:42.339 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57169.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57169.log.gz 2024-08-25T21:09:42.339 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52117.log 2024-08-25T21:09:42.339 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27135.log.gz 2024-08-25T21:09:42.340 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123813.log 2024-08-25T21:09:42.340 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.52117.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52117.log.gz 2024-08-25T21:09:42.340 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26251.log 2024-08-25T21:09:42.341 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123813.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123813.log.gz 2024-08-25T21:09:42.341 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121551.log 2024-08-25T21:09:42.342 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26251.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26251.log.gz 2024-08-25T21:09:42.342 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101302.log 2024-08-25T21:09:42.343 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121551.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121551.log.gz 2024-08-25T21:09:42.343 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91468.log 2024-08-25T21:09:42.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101302.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101302.log.gz 2024-08-25T21:09:42.344 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126987.log 2024-08-25T21:09:42.344 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91468.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91468.log.gz 2024-08-25T21:09:42.344 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73535.log 2024-08-25T21:09:42.345 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.126987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126987.log.gz 2024-08-25T21:09:42.345 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75051.log 2024-08-25T21:09:42.346 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73535.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73535.log.gz 2024-08-25T21:09:42.346 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30117.log 2024-08-25T21:09:42.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.75051.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75051.log.gz 2024-08-25T21:09:42.347 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126191.log 2024-08-25T21:09:42.347 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30117.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30117.log.gz 2024-08-25T21:09:42.348 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119564.log 2024-08-25T21:09:42.348 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.126191.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126191.log.gz 2024-08-25T21:09:42.349 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113444.log 2024-08-25T21:09:42.349 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102339.log 2024-08-25T21:09:42.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119564.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119564.log.gz 2024-08-25T21:09:42.350 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113444.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113444.log.gz 2024-08-25T21:09:42.350 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114446.log 2024-08-25T21:09:42.351 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102339.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102339.log.gz 2024-08-25T21:09:42.351 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111636.log 2024-08-25T21:09:42.352 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114446.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114446.log.gz 2024-08-25T21:09:42.352 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109614.log 2024-08-25T21:09:42.353 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111636.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111636.log.gz 2024-08-25T21:09:42.353 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131429.log 2024-08-25T21:09:42.354 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109614.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109614.log.gz 2024-08-25T21:09:42.354 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117758.log 2024-08-25T21:09:42.354 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.131429.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131429.log.gz 2024-08-25T21:09:42.354 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73644.log 2024-08-25T21:09:42.355 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.117758.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117758.log.gz 2024-08-25T21:09:42.355 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134930.log 2024-08-25T21:09:42.356 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73644.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73644.log.gz 2024-08-25T21:09:42.356 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82471.log 2024-08-25T21:09:42.357 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134930.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134930.log.gz 2024-08-25T21:09:42.357 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65632.log 2024-08-25T21:09:42.358 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.82471.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82471.log.gz 2024-08-25T21:09:42.358 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76791.log 2024-08-25T21:09:42.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65632.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65632.log.gz 2024-08-25T21:09:42.359 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73854.log 2024-08-25T21:09:42.359 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76791.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76791.log.gz 2024-08-25T21:09:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135641.log 2024-08-25T21:09:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73854.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73854.log.gz 2024-08-25T21:09:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109128.log 2024-08-25T21:09:42.361 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135641.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135641.log.gz 2024-08-25T21:09:42.361 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83269.log 2024-08-25T21:09:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109128.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109128.log.gz 2024-08-25T21:09:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81921.log 2024-08-25T21:09:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83269.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83269.log.gz -5 2024-08-25T21:09:42.363 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.39000.log 2024-08-25T21:09:42.363 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.81921.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81921.log.gz 2024-08-25T21:09:42.363 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113123.log 2024-08-25T21:09:42.364 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39000.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39000.log.gz 2024-08-25T21:09:42.364 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74927.log 2024-08-25T21:09:42.365 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113123.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113123.log.gz 2024-08-25T21:09:42.365 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135357.log 2024-08-25T21:09:42.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74927.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74927.log.gz 2024-08-25T21:09:42.366 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25935.log 2024-08-25T21:09:42.366 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135357.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135357.log.gz 2024-08-25T21:09:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122406.log 2024-08-25T21:09:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25935.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25935.log.gz 2024-08-25T21:09:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82658.log 2024-08-25T21:09:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122406.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122406.log.gz 2024-08-25T21:09:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25901.log 2024-08-25T21:09:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82658.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82658.log.gz 2024-08-25T21:09:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60607.log 2024-08-25T21:09:42.370 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25901.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25901.log.gz 2024-08-25T21:09:42.370 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65664.log 2024-08-25T21:09:42.371 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60607.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.admin.115102.log -- replaced with /var/log/ceph/cluster1-client.mirror.60607.log.gz 2024-08-25T21:09:42.371 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:42.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65664.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65664.log.gz 2024-08-25T21:09:42.372 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76392.log 2024-08-25T21:09:42.372 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115102.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135284.log 2024-08-25T21:09:42.372 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115102.log.gz 2024-08-25T21:09:42.373 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76392.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76392.log.gz 2024-08-25T21:09:42.373 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45839.log 2024-08-25T21:09:42.374 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135284.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74359.log 2024-08-25T21:09:42.374 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135284.log.gz 2024-08-25T21:09:42.375 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.45839.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45839.log.gz 2024-08-25T21:09:42.375 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113631.log 2024-08-25T21:09:42.376 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106375.log 2024-08-25T21:09:42.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74359.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74359.log.gz 2024-08-25T21:09:42.376 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.113631.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113631.log.gz 2024-08-25T21:09:42.377 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141718.log 2024-08-25T21:09:42.377 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.106375.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38576.log 2024-08-25T21:09:42.377 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106375.log.gz 2024-08-25T21:09:42.378 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141718.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141718.log.gz 2024-08-25T21:09:42.378 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135201.log 2024-08-25T21:09:42.379 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38576.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.38576.log.gz -- 2024-08-25T21:09:42.379 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.65296.log 2024-08-25T21:09:42.380 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135201.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135201.log.gz 2024-08-25T21:09:42.380 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80123.log 2024-08-25T21:09:42.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65296.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65296.log.gz 2024-08-25T21:09:42.381 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84784.log 2024-08-25T21:09:42.381 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80123.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38219.log 2024-08-25T21:09:42.382 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80123.log.gz 2024-08-25T21:09:42.382 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84784.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84784.log.gz 2024-08-25T21:09:42.382 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39424.log 2024-08-25T21:09:42.383 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38219.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38219.log.gz 2024-08-25T21:09:42.383 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114054.log 2024-08-25T21:09:42.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39424.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55732.log 2024-08-25T21:09:42.384 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39424.log.gz 2024-08-25T21:09:42.384 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114054.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114054.log.gz 2024-08-25T21:09:42.385 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98593.log 2024-08-25T21:09:42.385 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55732.log.gz 2024-08-25T21:09:42.386 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124864.log 2024-08-25T21:09:42.386 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98593.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98593.log.gz 2024-08-25T21:09:42.386 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2.log 2024-08-25T21:09:42.387 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124864.log.gz 2024-08-25T21:09:42.387 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127010.log 2024-08-25T21:09:42.388 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98508.log 2024-08-25T21:09:42.389 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127010.log.gz 2024-08-25T21:09:42.389 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77612.log 2024-08-25T21:09:42.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98508.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55841.log 2024-08-25T21:09:42.390 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98508.log.gz 2024-08-25T21:09:42.390 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77612.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77612.log.gz 2024-08-25T21:09:42.391 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107899.log 2024-08-25T21:09:42.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55841.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55841.log.gz 2024-08-25T21:09:42.392 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59379.log 2024-08-25T21:09:42.392 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107899.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53294.log 2024-08-25T21:09:42.393 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107899.log.gz 2024-08-25T21:09:42.393 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.59379.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59379.log.gz 2024-08-25T21:09:42.393 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66767.log 2024-08-25T21:09:42.394 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53294.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53294.log.gz 2024-08-25T21:09:42.394 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116790.log 2024-08-25T21:09:42.395 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66767.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66767.log.gz 2024-08-25T21:09:42.395 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103142.log 2024-08-25T21:09:42.396 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.116790.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116790.log.gz 2024-08-25T21:09:42.396 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61906.log 2024-08-25T21:09:42.397 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103142.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103142.log.gz 2024-08-25T21:09:42.397 INFO:teuthology.orchestra.run.smithi121.stderr: 87.9% -- replaced with /var/log/ceph/cluster2.log.gz 2024-08-25T21:09:42.397 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66629.log 2024-08-25T21:09:42.398 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61906.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.67428.log 2024-08-25T21:09:42.398 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.61906.log.gz 2024-08-25T21:09:42.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66629.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66629.log.gz 2024-08-25T21:09:42.399 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80007.log 2024-08-25T21:09:42.399 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67428.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49135.log 2024-08-25T21:09:42.400 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67428.log.gz 2024-08-25T21:09:42.400 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.80007.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80007.log.gz 2024-08-25T21:09:42.400 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51992.log 2024-08-25T21:09:42.401 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49135.log.gz 2024-08-25T21:09:42.401 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26841.log 2024-08-25T21:09:42.402 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51992.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51992.log.gz 2024-08-25T21:09:42.402 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.57970.log 2024-08-25T21:09:42.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26841.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26841.log.gz 2024-08-25T21:09:42.403 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65793.log 2024-08-25T21:09:42.403 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.57970.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118556.log 2024-08-25T21:09:42.403 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.57970.log.gz 2024-08-25T21:09:42.404 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65793.log.gz 2024-08-25T21:09:42.404 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53418.log 2024-08-25T21:09:42.405 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.118556.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118556.log.gz 2024-08-25T21:09:42.405 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94728.log 2024-08-25T21:09:42.406 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53418.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53418.log.gz 2024-08-25T21:09:42.406 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64530.log 2024-08-25T21:09:42.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.94728.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.70733.log -- replaced with /var/log/ceph/cluster1-client.mirror.94728.log.gz 2024-08-25T21:09:42.407 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:42.407 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64530.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123732.log 2024-08-25T21:09:42.407 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64530.log.gz 2024-08-25T21:09:42.408 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.70733.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70733.log.gz 2024-08-25T21:09:42.408 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46816.log 2024-08-25T21:09:42.409 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123732.log.gz 2024-08-25T21:09:42.409 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129115.log 2024-08-25T21:09:42.410 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46816.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46816.log.gz 2024-08-25T21:09:42.410 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99666.log 2024-08-25T21:09:42.411 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.129115.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129115.log.gz 2024-08-25T21:09:42.411 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111703.log 2024-08-25T21:09:42.411 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42187.log 2024-08-25T21:09:42.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99666.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99666.log.gz 2024-08-25T21:09:42.412 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111703.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111703.log.gz 2024-08-25T21:09:42.412 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103758.log 2024-08-25T21:09:42.413 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.42187.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88710.log 2024-08-25T21:09:42.413 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42187.log.gz 2024-08-25T21:09:42.414 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103758.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103758.log.gz 2024-08-25T21:09:42.414 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110034.log 2024-08-25T21:09:42.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88710.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.88710.log.gz --verbose 2024-08-25T21:09:42.415 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.admin.31106.log 2024-08-25T21:09:42.415 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110034.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110034.log.gz 2024-08-25T21:09:42.416 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101534.log 2024-08-25T21:09:42.416 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31106.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.admin.31106.log.gz -- 2024-08-25T21:09:42.416 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.50564.log 2024-08-25T21:09:42.417 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101534.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101534.log.gz 2024-08-25T21:09:42.417 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96619.log 2024-08-25T21:09:42.418 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50564.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50564.log.gz 2024-08-25T21:09:42.418 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104974.log 2024-08-25T21:09:42.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96619.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121923.log 2024-08-25T21:09:42.419 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96619.log.gz 2024-08-25T21:09:42.419 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104974.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104974.log.gz 2024-08-25T21:09:42.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109188.log 2024-08-25T21:09:42.420 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121923.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121923.log.gz 2024-08-25T21:09:42.420 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103451.log 2024-08-25T21:09:42.421 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109188.log.gz 2024-08-25T21:09:42.421 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77112.log 2024-08-25T21:09:42.422 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103451.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88059.log 2024-08-25T21:09:42.422 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103451.log.gz 2024-08-25T21:09:42.423 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77112.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77112.log.gz 2024-08-25T21:09:42.423 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101255.log 2024-08-25T21:09:42.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88059.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88059.log.gz 2024-08-25T21:09:42.424 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132731.log 2024-08-25T21:09:42.424 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101255.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101255.log.gz 2024-08-25T21:09:42.424 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60964.log 2024-08-25T21:09:42.425 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132731.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27598.log 2024-08-25T21:09:42.425 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132731.log.gz 2024-08-25T21:09:42.426 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60964.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60964.log.gz 2024-08-25T21:09:42.426 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29140.log 2024-08-25T21:09:42.427 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27598.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27598.log.gz 2024-08-25T21:09:42.427 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62536.log 2024-08-25T21:09:42.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29140.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29140.log.gz 2024-08-25T21:09:42.428 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84787.log 2024-08-25T21:09:42.428 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62536.log.gz 2024-08-25T21:09:42.429 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74253.log 2024-08-25T21:09:42.429 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84787.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123494.log 2024-08-25T21:09:42.429 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84787.log.gz 2024-08-25T21:09:42.430 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74253.log.gz 2024-08-25T21:09:42.430 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26190.log 2024-08-25T21:09:42.431 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46541.log 2024-08-25T21:09:42.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123494.log.gz 2024-08-25T21:09:42.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26190.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92589.log 2024-08-25T21:09:42.432 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26190.log.gz 2024-08-25T21:09:42.432 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.46541.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46541.log.gz 2024-08-25T21:09:42.433 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37150.log 2024-08-25T21:09:42.433 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92589.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92589.log.gz 2024-08-25T21:09:42.433 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100017.log 2024-08-25T21:09:42.434 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37150.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37150.log.gz 2024-08-25T21:09:42.434 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47826.log 2024-08-25T21:09:42.435 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100017.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100017.log.gz 2024-08-25T21:09:42.435 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62620.log 2024-08-25T21:09:42.436 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47826.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47826.log.gz 2024-08-25T21:09:42.436 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101365.log 2024-08-25T21:09:42.437 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62620.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66317.log 2024-08-25T21:09:42.437 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62620.log.gz 2024-08-25T21:09:42.438 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101365.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101365.log.gz 2024-08-25T21:09:42.438 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121120.log 2024-08-25T21:09:42.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66317.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66317.log.gz 2024-08-25T21:09:42.439 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112121.log 2024-08-25T21:09:42.439 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.121120.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121120.log.gz 2024-08-25T21:09:42.440 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63288.log 2024-08-25T21:09:42.440 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.112121.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57553.log 2024-08-25T21:09:42.440 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112121.log.gz 2024-08-25T21:09:42.441 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63288.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63288.log.gz 2024-08-25T21:09:42.441 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27177.log 2024-08-25T21:09:42.442 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57553.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57553.log.gz 2024-08-25T21:09:42.442 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134865.log 2024-08-25T21:09:42.443 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27177.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27177.log.gz 2024-08-25T21:09:42.443 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95102.log 2024-08-25T21:09:42.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134865.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134865.log.gz 2024-08-25T21:09:42.444 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63124.log 2024-08-25T21:09:42.444 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95102.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29402.log 0.0% 2024-08-25T21:09:42.445 INFO:teuthology.orchestra.run.smithi121.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.95102.log.gz 2024-08-25T21:09:42.445 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.63124.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63124.log.gz 2024-08-25T21:09:42.445 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40989.log 2024-08-25T21:09:42.446 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29402.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29402.log.gz 2024-08-25T21:09:42.446 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111343.log 2024-08-25T21:09:42.447 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40989.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40989.log.gz 2024-08-25T21:09:42.447 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69185.log 2024-08-25T21:09:42.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111343.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111343.log.gz 2024-08-25T21:09:42.448 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139121.log 2024-08-25T21:09:42.448 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69185.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69185.log.gz 2024-08-25T21:09:42.449 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137933.log 2024-08-25T21:09:42.449 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139121.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139121.log.gz 2024-08-25T21:09:42.450 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76060.log 2024-08-25T21:09:42.450 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.137933.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137933.log.gz 2024-08-25T21:09:42.451 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118609.log 2024-08-25T21:09:42.451 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76060.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76060.log.gz 2024-08-25T21:09:42.451 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77469.log 2024-08-25T21:09:42.452 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.118609.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118609.log.gz 2024-08-25T21:09:42.452 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.22181.log 2024-08-25T21:09:42.453 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77469.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77469.log.gz 2024-08-25T21:09:42.453 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26294.log 2024-08-25T21:09:42.454 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.22181.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127319.log 2024-08-25T21:09:42.454 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.22181.log.gz 2024-08-25T21:09:42.455 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26294.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26294.log.gz 2024-08-25T21:09:42.455 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89537.log 2024-08-25T21:09:42.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127319.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78442.log 2024-08-25T21:09:42.456 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127319.log.gz 2024-08-25T21:09:42.456 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89537.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89537.log.gz 2024-08-25T21:09:42.456 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58814.log 2024-08-25T21:09:42.457 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78442.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78442.log.gz 2024-08-25T21:09:42.457 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122364.log 2024-08-25T21:09:42.458 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58814.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118778.log 2024-08-25T21:09:42.458 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58814.log.gz 2024-08-25T21:09:42.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122364.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122364.log.gz 2024-08-25T21:09:42.459 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55512.log 2024-08-25T21:09:42.459 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.118778.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118778.log.gz 2024-08-25T21:09:42.460 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48017.log 2024-08-25T21:09:42.460 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55512.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55512.log.gz 2024-08-25T21:09:42.460 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135283.log 2024-08-25T21:09:42.461 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.48017.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48017.log.gz 2024-08-25T21:09:42.461 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101722.log 2024-08-25T21:09:42.462 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135283.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135283.log.gz 2024-08-25T21:09:42.462 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85160.log 2024-08-25T21:09:42.463 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101722.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101722.log.gz 2024-08-25T21:09:42.463 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76270.log 2024-08-25T21:09:42.464 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85160.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85160.log.gz 2024-08-25T21:09:42.464 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31500.log 2024-08-25T21:09:42.465 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76270.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46789.log 2024-08-25T21:09:42.465 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76270.log.gz 2024-08-25T21:09:42.465 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.31500.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31500.log.gz 2024-08-25T21:09:42.466 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129409.log 2024-08-25T21:09:42.466 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46789.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46789.log.gz 2024-08-25T21:09:42.466 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124234.log 2024-08-25T21:09:42.467 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129409.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129409.log.gz 2024-08-25T21:09:42.467 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62808.log 2024-08-25T21:09:42.468 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124234.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124234.log.gz 2024-08-25T21:09:42.468 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96457.log 2024-08-25T21:09:42.469 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.62808.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62808.log.gz 2024-08-25T21:09:42.469 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44436.log 2024-08-25T21:09:42.470 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96457.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96457.log.gz 2024-08-25T21:09:42.470 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103184.log 2024-08-25T21:09:42.471 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44436.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44436.log.gz 2024-08-25T21:09:42.471 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109130.log 2024-08-25T21:09:42.472 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103184.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103184.log.gz 2024-08-25T21:09:42.472 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.29742.log 2024-08-25T21:09:42.472 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109130.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109130.log.gz 2024-08-25T21:09:42.473 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117280.log 2024-08-25T21:09:42.473 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.29742.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.29742.log.gz 2024-08-25T21:09:42.473 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94118.log 2024-08-25T21:09:42.474 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.117280.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117280.log.gz 2024-08-25T21:09:42.474 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57698.log 2024-08-25T21:09:42.475 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.94118.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94118.log.gz 2024-08-25T21:09:42.475 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.143093.log 2024-08-25T21:09:42.476 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57698.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84159.log 2024-08-25T21:09:42.476 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57698.log.gz 2024-08-25T21:09:42.476 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.143093.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.143093.log.gz 2024-08-25T21:09:42.477 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66612.log 2024-08-25T21:09:42.477 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84159.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84159.log.gz 2024-08-25T21:09:42.478 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84889.log 2024-08-25T21:09:42.478 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.66612.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66612.log.gz 2024-08-25T21:09:42.478 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100650.log 2024-08-25T21:09:42.479 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84889.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84889.log.gz 2024-08-25T21:09:42.479 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39415.log 2024-08-25T21:09:42.480 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.100650.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35317.log 2024-08-25T21:09:42.480 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100650.log.gz 2024-08-25T21:09:42.481 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.39415.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39415.log.gz 2024-08-25T21:09:42.481 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32203.log 2024-08-25T21:09:42.481 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35317.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35317.log.gz 2024-08-25T21:09:42.481 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115328.log 2024-08-25T21:09:42.482 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.32203.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117323.log 2024-08-25T21:09:42.482 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32203.log.gz 2024-08-25T21:09:42.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115328.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.115328.log.gz 2024-08-25T21:09:42.483 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49992.log 2024-08-25T21:09:42.483 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117323.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117323.log.gz 2024-08-25T21:09:42.484 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117051.log 2024-08-25T21:09:42.484 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49992.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.49992.log.gz 2024-08-25T21:09:42.484 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.87998.log 2024-08-25T21:09:42.485 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.117051.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117051.log.gz 2024-08-25T21:09:42.485 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44664.log 2024-08-25T21:09:42.486 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87998.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87998.log.gz 2024-08-25T21:09:42.486 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96281.log 2024-08-25T21:09:42.487 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.44664.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44664.log.gz 2024-08-25T21:09:42.487 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47892.log 2024-08-25T21:09:42.487 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96281.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96281.log.gz 2024-08-25T21:09:42.488 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103141.log 2024-08-25T21:09:42.488 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47892.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47892.log.gz 2024-08-25T21:09:42.488 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79791.log 2024-08-25T21:09:42.489 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.103141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103141.log.gz 2024-08-25T21:09:42.489 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89949.log 2024-08-25T21:09:42.490 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79791.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79791.log.gz 2024-08-25T21:09:42.490 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120602.log 2024-08-25T21:09:42.491 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.89949.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.89949.log.gz -- 2024-08-25T21:09:42.491 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.66253.log 2024-08-25T21:09:42.491 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120602.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120602.log.gz 2024-08-25T21:09:42.491 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49912.log 2024-08-25T21:09:42.492 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.66253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66253.log.gz 2024-08-25T21:09:42.493 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27091.log 2024-08-25T21:09:42.493 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49912.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49912.log.gz 2024-08-25T21:09:42.493 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116378.log 2024-08-25T21:09:42.494 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27091.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27091.log.gz 2024-08-25T21:09:42.494 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111933.log 2024-08-25T21:09:42.495 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116378.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116378.log.gz 2024-08-25T21:09:42.495 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104894.log 2024-08-25T21:09:42.495 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.111933.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111933.log.gz 2024-08-25T21:09:42.496 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81681.log 2024-08-25T21:09:42.496 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104894.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104894.log.gz 2024-08-25T21:09:42.496 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80471.log 2024-08-25T21:09:42.497 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81681.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81681.log.gz 2024-08-25T21:09:42.497 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35389.log 2024-08-25T21:09:42.498 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.80471.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80471.log.gz 2024-08-25T21:09:42.498 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128396.log 2024-08-25T21:09:42.499 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35389.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35389.log.gz 2024-08-25T21:09:42.499 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84617.log 2024-08-25T21:09:42.500 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128396.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128396.log.gz 2024-08-25T21:09:42.500 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97966.log 2024-08-25T21:09:42.501 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.84617.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84617.log.gz 2024-08-25T21:09:42.501 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28983.log 2024-08-25T21:09:42.501 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97966.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.97966.log.gz 2024-08-25T21:09:42.502 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.139407.log 2024-08-25T21:09:42.502 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28983.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28983.log.gz 2024-08-25T21:09:42.502 INFO:teuthology.orchestra.run.smithi121.stderr: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.0.22426.log.gz 2024-08-25T21:09:42.502 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109066.log 2024-08-25T21:09:42.503 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139407.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139407.log.gz 2024-08-25T21:09:42.503 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85706.log 2024-08-25T21:09:42.504 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109066.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109066.log.gz 2024-08-25T21:09:42.504 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101154.log 2024-08-25T21:09:42.505 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85706.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57996.log 2024-08-25T21:09:42.505 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85706.log.gz 2024-08-25T21:09:42.505 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.101154.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101154.log.gz 2024-08-25T21:09:42.506 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54343.log 2024-08-25T21:09:42.506 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.57996.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57996.log.gz 2024-08-25T21:09:42.506 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46862.log 2024-08-25T21:09:42.507 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54343.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54343.log.gz 2024-08-25T21:09:42.507 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61215.log 2024-08-25T21:09:42.508 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.46862.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54646.log 2024-08-25T21:09:42.508 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46862.log.gz 2024-08-25T21:09:42.509 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61215.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61215.log.gz 2024-08-25T21:09:42.509 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36645.log 2024-08-25T21:09:42.509 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.54646.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54646.log.gz 2024-08-25T21:09:42.510 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47389.log 2024-08-25T21:09:42.510 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.36645.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36645.log.gz 2024-08-25T21:09:42.511 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129812.log 2024-08-25T21:09:42.511 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47389.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47389.log.gz 2024-08-25T21:09:42.511 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130687.log 2024-08-25T21:09:42.512 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.129812.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.74713.log -- replaced with /var/log/ceph/cluster1-client.admin.129812.log.gz 2024-08-25T21:09:42.512 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:09:42.513 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.130687.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130687.log.gzgzip 2024-08-25T21:09:42.513 INFO:teuthology.orchestra.run.smithi121.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82564.log 2024-08-25T21:09:42.513 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74713.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74713.log.gz -5 2024-08-25T21:09:42.513 INFO:teuthology.orchestra.run.smithi121.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.123312.log 2024-08-25T21:09:42.514 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.82564.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82564.log.gz 2024-08-25T21:09:42.514 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98845.log 2024-08-25T21:09:42.515 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.123312.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123312.log.gz 2024-08-25T21:09:42.518 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41558.log 2024-08-25T21:09:42.521 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98845.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73348.log 2024-08-25T21:09:42.521 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98845.log.gz 2024-08-25T21:09:42.521 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41558.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70143.log 2024-08-25T21:09:42.521 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41558.log.gz 2024-08-25T21:09:42.521 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73348.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73348.log.gz 2024-08-25T21:09:42.521 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102732.log 2024-08-25T21:09:42.521 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.70143.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70143.log.gz 2024-08-25T21:09:42.521 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139253.log 2024-08-25T21:09:42.522 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.102732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102732.log.gz 2024-08-25T21:09:42.522 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40549.log 2024-08-25T21:09:42.522 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.139253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139253.log.gz 2024-08-25T21:09:42.522 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54094.log 2024-08-25T21:09:42.522 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40549.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132084.log 2024-08-25T21:09:42.522 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40549.log.gz 2024-08-25T21:09:42.523 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.54094.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49849.log 2024-08-25T21:09:42.523 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54094.log.gz 2024-08-25T21:09:42.523 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.132084.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132084.log.gz 2024-08-25T21:09:42.523 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128609.log 2024-08-25T21:09:42.523 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49849.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49849.log.gz 2024-08-25T21:09:42.524 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61843.log 2024-08-25T21:09:42.525 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.128609.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40231.log 2024-08-25T21:09:42.525 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128609.log.gz 2024-08-25T21:09:42.525 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61843.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61843.log.gz 2024-08-25T21:09:42.525 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109991.log 2024-08-25T21:09:42.526 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40231.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40231.log.gz 2024-08-25T21:09:42.526 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119537.log 2024-08-25T21:09:42.526 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109991.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109991.log.gz 2024-08-25T21:09:42.526 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136814.log 2024-08-25T21:09:42.527 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.119537.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119537.log.gz 2024-08-25T21:09:42.527 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81001.log 2024-08-25T21:09:42.528 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136814.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.136814.log.gz --verbose 2024-08-25T21:09:42.528 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.41809.log 2024-08-25T21:09:42.529 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.81001.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81001.log.gz 2024-08-25T21:09:42.529 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58627.log 2024-08-25T21:09:42.530 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.41809.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41809.log.gz 2024-08-25T21:09:42.530 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34552.log 2024-08-25T21:09:42.530 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.58627.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58627.log.gz 2024-08-25T21:09:42.531 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77680.log 2024-08-25T21:09:42.531 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.34552.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34552.log.gz 2024-08-25T21:09:42.531 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90226.log 2024-08-25T21:09:42.532 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77680.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77680.log.gz 2024-08-25T21:09:42.532 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.26942.log 2024-08-25T21:09:42.533 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.90226.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90226.log.gz 2024-08-25T21:09:42.533 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141289.log 2024-08-25T21:09:42.534 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.26942.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.26942.log.gz 2024-08-25T21:09:42.534 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74529.log 2024-08-25T21:09:42.535 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.141289.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141289.log.gz 2024-08-25T21:09:42.535 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27782.log 2024-08-25T21:09:42.536 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74529.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74529.log.gz 2024-08-25T21:09:42.536 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.30032.log 2024-08-25T21:09:42.536 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27782.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27782.log.gz 2024-08-25T21:09:42.537 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25591.log 2024-08-25T21:09:42.537 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.30032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.30032.log.gz 2024-08-25T21:09:42.537 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99262.log 2024-08-25T21:09:42.538 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25591.log.gz 2024-08-25T21:09:42.538 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97185.log 2024-08-25T21:09:42.539 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.99262.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99262.log.gz 2024-08-25T21:09:42.539 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96126.log 2024-08-25T21:09:42.540 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97185.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97185.log.gz 2024-08-25T21:09:42.540 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78740.log 2024-08-25T21:09:42.540 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96126.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96126.log.gz 2024-08-25T21:09:42.540 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.142113.log 2024-08-25T21:09:42.541 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.78740.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78740.log.gz 2024-08-25T21:09:42.541 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65456.log 2024-08-25T21:09:42.542 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.142113.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.142113.log.gz 2024-08-25T21:09:42.542 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95885.log 2024-08-25T21:09:42.543 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65456.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65456.log.gz 2024-08-25T21:09:42.543 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137979.log 2024-08-25T21:09:42.544 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95885.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95885.log.gz 2024-08-25T21:09:42.544 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83795.log 2024-08-25T21:09:42.544 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137979.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137979.log.gz 2024-08-25T21:09:42.545 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136295.log 2024-08-25T21:09:42.545 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.83795.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83795.log.gz 2024-08-25T21:09:42.546 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96494.log 2024-08-25T21:09:42.546 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.136295.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136295.log.gz 2024-08-25T21:09:42.547 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115584.log 2024-08-25T21:09:42.547 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.96494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96494.log.gz 2024-08-25T21:09:42.548 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61884.log 2024-08-25T21:09:42.548 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.115584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115584.log.gz 2024-08-25T21:09:42.548 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124521.log 2024-08-25T21:09:42.549 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61884.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61884.log.gz 2024-08-25T21:09:42.549 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105260.log 2024-08-25T21:09:42.550 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.124521.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124521.log.gz 2024-08-25T21:09:42.550 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95539.log 2024-08-25T21:09:42.551 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.105260.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster2-client.admin.105260.log.gz --verbose -- 2024-08-25T21:09:42.551 INFO:teuthology.orchestra.run.smithi121.stderr: /var/log/ceph/cluster1-client.mirror.60122.log 2024-08-25T21:09:42.551 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.95539.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95539.log.gz 2024-08-25T21:09:42.552 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25691.log 2024-08-25T21:09:42.552 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60122.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79482.log 2024-08-25T21:09:42.552 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60122.log.gz 2024-08-25T21:09:42.553 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25691.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87684.log 2024-08-25T21:09:42.553 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25691.log.gz 2024-08-25T21:09:42.554 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.79482.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79482.log.gz 2024-08-25T21:09:42.554 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68653.log 2024-08-25T21:09:42.555 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87684.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87684.log.gz 2024-08-25T21:09:42.555 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55422.log 2024-08-25T21:09:42.555 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68653.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126412.log 2024-08-25T21:09:42.555 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68653.log.gz 2024-08-25T21:09:42.556 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.55422.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55422.log.gz 2024-08-25T21:09:42.556 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106676.log 2024-08-25T21:09:42.557 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.126412.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126412.log.gz 2024-08-25T21:09:42.557 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88363.log 2024-08-25T21:09:42.558 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.106676.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106676.log.gz 2024-08-25T21:09:42.558 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78600.log 2024-08-25T21:09:42.558 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.88363.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88363.log.gz 2024-08-25T21:09:42.558 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125704.log 2024-08-25T21:09:42.559 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.78600.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78600.log.gz 2024-08-25T21:09:42.559 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97045.log 2024-08-25T21:09:42.560 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.125704.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125704.log.gz 2024-08-25T21:09:42.560 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129315.log 2024-08-25T21:09:42.561 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97045.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97045.log.gz 2024-08-25T21:09:42.561 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114397.log 2024-08-25T21:09:42.561 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.129315.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129315.log.gz 2024-08-25T21:09:42.562 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87017.log 2024-08-25T21:09:42.563 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.114397.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114397.log.gz 2024-08-25T21:09:42.563 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65330.log 2024-08-25T21:09:42.563 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87017.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87017.log.gz 2024-08-25T21:09:42.564 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110550.log 2024-08-25T21:09:42.564 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.65330.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65330.log.gz 2024-08-25T21:09:42.565 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.27362.log 2024-08-25T21:09:42.565 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.110550.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110550.log.gz 2024-08-25T21:09:42.565 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40509.log 2024-08-25T21:09:42.566 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.27362.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85079.log 2024-08-25T21:09:42.566 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.27362.log.gz 2024-08-25T21:09:42.567 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.40509.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40509.log.gz 2024-08-25T21:09:42.567 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82518.log 2024-08-25T21:09:42.568 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.85079.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85079.log.gz 2024-08-25T21:09:42.568 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69652.log 2024-08-25T21:09:42.568 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.82518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82518.log.gz 2024-08-25T21:09:42.568 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58151.log 2024-08-25T21:09:42.569 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.69652.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69652.log.gz 2024-08-25T21:09:42.569 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35812.log 2024-08-25T21:09:42.570 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.58151.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58151.log.gz 2024-08-25T21:09:42.570 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60671.log 2024-08-25T21:09:42.571 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.35812.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77572.log 2024-08-25T21:09:42.571 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35812.log.gz 2024-08-25T21:09:42.572 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.60671.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60671.log.gz 2024-08-25T21:09:42.572 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93130.log 2024-08-25T21:09:42.572 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.77572.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77572.log.gz 2024-08-25T21:09:42.572 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.32845.log 2024-08-25T21:09:42.573 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.93130.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93130.log.gz 2024-08-25T21:09:42.573 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73306.log 2024-08-25T21:09:42.574 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.32845.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32845.log.gz 2024-08-25T21:09:42.574 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76312.log 2024-08-25T21:09:42.575 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.73306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73306.log.gz 2024-08-25T21:09:42.575 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51696.log 2024-08-25T21:09:42.575 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.76312.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76312.log.gz 2024-08-25T21:09:42.576 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72590.log 2024-08-25T21:09:42.576 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.51696.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51696.log.gz 2024-08-25T21:09:42.576 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61306.log 2024-08-25T21:09:42.577 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.72590.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72590.log.gz 2024-08-25T21:09:42.577 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127428.log 2024-08-25T21:09:42.578 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61306.log.gz 2024-08-25T21:09:42.578 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92982.log 2024-08-25T21:09:42.579 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.127428.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127428.log.gz 2024-08-25T21:09:42.579 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47584.log 2024-08-25T21:09:42.579 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.92982.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster2-client.admin.92982.log.gz --verbose 2024-08-25T21:09:42.580 INFO:teuthology.orchestra.run.smithi121.stderr: -- /var/log/ceph/cluster1-client.mirror.25378.log 2024-08-25T21:09:42.580 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.47584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47584.log.gz 2024-08-25T21:09:42.580 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68532.log 2024-08-25T21:09:42.581 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25378.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25378.log.gz 2024-08-25T21:09:42.581 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50772.log 2024-08-25T21:09:42.582 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.68532.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68532.log.gz 2024-08-25T21:09:42.582 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116478.log 2024-08-25T21:09:42.583 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.50772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50772.log.gz 2024-08-25T21:09:42.583 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58016.log 2024-08-25T21:09:42.583 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.116478.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116478.log.gz 2024-08-25T21:09:42.584 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46219.log 2024-08-25T21:09:42.585 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.58016.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91226.log 2024-08-25T21:09:42.585 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58016.log.gz 2024-08-25T21:09:42.585 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.46219.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46219.log.gz 2024-08-25T21:09:42.586 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37358.log 2024-08-25T21:09:42.586 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91226.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91226.log.gz 2024-08-25T21:09:42.586 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57172.log 2024-08-25T21:09:42.587 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37358.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37358.log.gz 2024-08-25T21:09:42.587 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137532.log 2024-08-25T21:09:42.588 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.57172.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57172.log.gz 2024-08-25T21:09:42.588 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61256.log 2024-08-25T21:09:42.589 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137532.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137532.log.gz 2024-08-25T21:09:42.589 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104221.log 2024-08-25T21:09:42.589 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.61256.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61256.log.gz 2024-08-25T21:09:42.590 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93776.log 2024-08-25T21:09:42.591 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64239.log 2024-08-25T21:09:42.591 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.104221.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104221.log.gz 2024-08-25T21:09:42.591 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.93776.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93776.log.gz 2024-08-25T21:09:42.591 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74611.log 2024-08-25T21:09:42.592 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.64239.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64239.log.gz 2024-08-25T21:09:42.592 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135996.log 2024-08-25T21:09:42.593 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.74611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74611.log.gz 2024-08-25T21:09:42.593 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107717.log 2024-08-25T21:09:42.594 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.135996.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109391.log 2024-08-25T21:09:42.594 INFO:teuthology.orchestra.run.smithi121.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135996.log.gz 2024-08-25T21:09:42.594 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.107717.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107717.log.gz 2024-08-25T21:09:42.595 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49428.log 2024-08-25T21:09:42.595 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.109391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109391.log.gz 2024-08-25T21:09:42.595 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.28902.log 2024-08-25T21:09:42.596 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.49428.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49428.log.gz 2024-08-25T21:09:42.596 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53292.log 2024-08-25T21:09:42.597 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.28902.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.28902.log.gz 2024-08-25T21:09:42.597 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67406.log 2024-08-25T21:09:42.598 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.53292.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53292.log.gz 2024-08-25T21:09:42.598 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91273.log 2024-08-25T21:09:42.598 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.67406.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67406.log.gz 2024-08-25T21:09:42.599 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.0.22425.log 2024-08-25T21:09:42.599 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.91273.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91273.log.gz 2024-08-25T21:09:42.599 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56381.log 2024-08-25T21:09:42.600 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.0.22425.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56662.log 2024-08-25T21:09:42.601 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.56381.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56381.log.gz 2024-08-25T21:09:42.601 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37276.log 2024-08-25T21:09:42.602 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.admin.56662.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56662.log.gz 2024-08-25T21:09:42.602 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120642.log 2024-08-25T21:09:42.602 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.37276.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37276.log.gz 2024-08-25T21:09:42.603 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134653.log 2024-08-25T21:09:42.603 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.120642.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120642.log.gz 2024-08-25T21:09:42.603 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122405.log 2024-08-25T21:09:42.604 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134653.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134653.log.gz 2024-08-25T21:09:42.604 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.25419.log 2024-08-25T21:09:42.605 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.122405.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122405.log.gz 2024-08-25T21:09:42.605 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87009.log 2024-08-25T21:09:42.605 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.25419.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.25419.log.gz 2024-08-25T21:09:42.606 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.33440.log 2024-08-25T21:09:42.606 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.87009.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87009.log.gz 2024-08-25T21:09:42.607 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92480.log 2024-08-25T21:09:42.607 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.33440.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.33440.log.gz 2024-08-25T21:09:42.607 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137164.log 2024-08-25T21:09:42.608 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.admin.92480.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92480.log.gz 2024-08-25T21:09:42.608 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134591.log 2024-08-25T21:09:42.609 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.137164.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137164.log.gz 2024-08-25T21:09:42.609 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98549.log 2024-08-25T21:09:42.610 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.134591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134591.log.gz 2024-08-25T21:09:42.610 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38661.log 2024-08-25T21:09:42.610 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.98549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98549.log.gz 2024-08-25T21:09:42.610 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45017.log 2024-08-25T21:09:42.611 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.38661.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38661.log.gz 2024-08-25T21:09:42.611 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97586.log 2024-08-25T21:09:42.612 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster2-client.mirror.45017.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45017.log.gz 2024-08-25T21:09:42.613 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cluster1-client.mirror.97586.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97586.log.gz 2024-08-25T21:09:44.118 INFO:teuthology.orchestra.run.smithi121.stderr: 82.3% -- replaced with /var/log/ceph/cluster1-client.mirror.0.22425.log.gz 2024-08-25T21:09:50.991 INFO:teuthology.orchestra.run.smithi121.stderr: 91.5% -- replaced with /var/log/ceph/cluster2-mgr.x.log.gz 2024-08-25T21:09:53.013 INFO:teuthology.orchestra.run.smithi119.stderr: 90.5% -- replaced with /var/log/ceph/cluster1-mon.a.log.gz 2024-08-25T21:10:10.237 INFO:teuthology.orchestra.run.smithi121.stderr: 90.2% -- replaced with /var/log/ceph/cluster2-mon.a.log.gz 2024-08-25T21:14:06.482 INFO:teuthology.orchestra.run.smithi121.stderr: 94.3% -- replaced with /var/log/ceph/cluster2-osd.1.log.gz 2024-08-25T21:14:14.055 INFO:teuthology.orchestra.run.smithi121.stderr: 94.3% -- replaced with /var/log/ceph/cluster2-osd.2.log.gz 2024-08-25T21:14:37.761 INFO:teuthology.orchestra.run.smithi121.stderr: 94.3% -- replaced with /var/log/ceph/cluster2-osd.0.log.gz 2024-08-25T21:14:37.764 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-08-25T21:14:37.764 INFO:teuthology.orchestra.run.smithi121.stderr:real 4m59.568s 2024-08-25T21:14:37.764 INFO:teuthology.orchestra.run.smithi121.stderr:user 2m6.018s 2024-08-25T21:14:37.764 INFO:teuthology.orchestra.run.smithi121.stderr:sys 0m29.563s 2024-08-25T21:18:51.231 INFO:teuthology.orchestra.run.smithi119.stderr: 94.2% -- replaced with /var/log/ceph/cluster1-osd.1.log.gz 2024-08-25T21:19:30.892 INFO:teuthology.orchestra.run.smithi119.stderr: 94.3% -- replaced with /var/log/ceph/cluster1-osd.2.log.gz 2024-08-25T21:19:41.027 INFO:teuthology.orchestra.run.smithi119.stderr: 94.2% -- replaced with /var/log/ceph/cluster1-osd.0.log.gz 2024-08-25T21:19:41.079 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-08-25T21:19:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:real 10m2.874s 2024-08-25T21:19:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:user 4m43.628s 2024-08-25T21:19:41.079 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m42.695s 2024-08-25T21:19:41.081 INFO:tasks.ceph:Archiving logs... 2024-08-25T21:19:41.081 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544/remote/smithi119/log 2024-08-25T21:19:41.082 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-25T21:20:21.220 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544/remote/smithi121/log 2024-08-25T21:20:21.221 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-25T21:20:44.098 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-25T21:20:44.129 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-25T21:20:44.129 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-25T21:20:44.134 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-25T21:20:44.259 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-25T21:20:44.269 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-25T21:20:44.270 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-25T21:20:44.275 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-25T21:20:44.307 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-08-25T21:20:44.311 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-08-25T21:20:44.425 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-25T21:20:44.425 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-08-25T21:20:44.425 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 +821us[ +807us] +/- 60ms 2024-08-25T21:20:44.425 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -1089us[-1089us] +/- 55ms 2024-08-25T21:20:44.425 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 68 +343us[ +333us] +/- 66ms 2024-08-25T21:20:44.425 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-08-25T21:20:44.435 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-25T21:20:44.436 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-08-25T21:20:44.436 INFO:teuthology.orchestra.run.smithi121.stdout:^* hv01.front.sepia.ceph.com 3 7 377 13 +769us[ +852us] +/- 60ms 2024-08-25T21:20:44.436 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -1173us[-1173us] +/- 55ms 2024-08-25T21:20:44.436 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +262us[ +262us] +/- 67ms 2024-08-25T21:20:44.436 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-08-25T21:20:44.437 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-25T21:20:44.447 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-25T21:20:44.447 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-25T21:20:44.456 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-25T21:20:44.583 DEBUG:teuthology.orchestra.run.smithi121:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-25T21:20:44.796 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-08-25T21:20:44.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-25T21:20:44.858 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-08-25T21:20:44.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-25T21:20:44.918 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-25T21:20:44.928 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-25T21:20:44.937 INFO:teuthology.task.internal:Duration was 5964.257664 seconds 2024-08-25T21:20:44.937 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-25T21:20:44.946 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-25T21:20:44.947 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-25T21:20:44.951 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-25T21:20:45.088 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-25T21:20:45.118 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-25T21:20:45.403 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-25T21:20:45.403 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-08-25T21:20:45.403 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-25T21:20:45.457 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2024-08-25T21:20:45.458 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-25T21:20:45.507 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-25T21:20:45.507 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-25T21:20:45.512 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-25T21:20:45.570 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-25T21:20:45.570 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-25T21:20:45.575 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-25T21:20:45.756 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-25T21:20:45.766 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-25T21:20:45.766 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-25T21:20:45.838 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-25T21:20:45.940 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-25T21:20:45.951 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-25T21:20:45.955 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-25T21:20:46.036 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2024-08-25T21:20:46.038 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-08-25T21:20:46.075 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-25T21:20:46.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-25T21:20:46.109 DEBUG:teuthology.orchestra.run.smithi121:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-25T21:20:46.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-25T21:20:46.142 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-25T21:20:46.152 INFO:teuthology.task.internal:Transferring archived files... 2024-08-25T21:20:46.153 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544/remote/smithi119 2024-08-25T21:20:46.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-25T21:20:46.212 DEBUG:teuthology.misc:Transferring archived files from smithi121:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-08-24_20:48:16-krbd-main-testing-default-smithi/7870544/remote/smithi121 2024-08-25T21:20:46.213 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-25T21:20:46.274 INFO:teuthology.task.internal:Removing archive directory... 2024-08-25T21:20:46.275 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-25T21:20:46.279 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-25T21:20:46.313 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-25T21:20:46.325 INFO:teuthology.task.internal:Not uploading archives. 2024-08-25T21:20:46.325 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-25T21:20:46.336 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-25T21:20:46.336 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-25T21:20:46.354 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-25T21:20:46.384 INFO:teuthology.orchestra.run.smithi119.stdout: 393246 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 25 21:20 /home/ubuntu/cephtest 2024-08-25T21:20:46.388 INFO:teuthology.orchestra.run.smithi121.stdout: 393721 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 25 21:20 /home/ubuntu/cephtest 2024-08-25T21:20:46.391 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-25T21:20:46.401 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-25T21:20:46.425 INFO:teuthology.run:Summary data: cluster1.mon.a-kernel-sha1: 8618c856c9a644ad80ca70c748bebe0c9b6227d4 cluster2.mon.a-kernel-sha1: 8618c856c9a644ad80ca70c748bebe0c9b6227d4 description: krbd/mirror/{bluestore-bitmap clusters/2-node conf install/ceph ms_mode$/{legacy} tasks/compare-mirror-images} duration: 5964.257664203644 failure_reason: '"2024-08-25T19:48:35.950872+0000 mon.a (mon.0) 206 : cluster [WRN] Health check failed: Reduced data availability: 3 pgs inactive, 3 pgs peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-08-25T21:20:46.425 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-25T21:20:46.513 INFO:teuthology.run:FAIL